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Preface 



The Earth, the Blue planet, is so called because its larger biotope is a vast and deep 
intertwined blue expanse of salted oceans and seas. The water is filled with billions 
of organisms which are mainly invisible to the naked eye. These life forms have 
allowed us to breathe and to conquer the lands. They are our ancestors, our stones, 
our oils, our food or the food of our food . . . and they may hold the key to our future 
in these troubled times of climate change. 

The idea for this book germinated during an oceanographic survey in the 
Mediterranean Sea where, as a cell biologist coming out of a high ranked institute 
of fundamental research, I realized that many imaging techniques including 3D 
microscopy techniques were not easily available to marine biologists. I came back 
eager to share my knowledge with this large and amazing scientific community. 
Many of my friends responded positively and shared the burden of tracking any 
application of advanced imaging methods in marine biology and I would like 
to thank them all, in particular Renaud Boistel, who managed a huge crowd of 
co-authors to give us a wonderful chapter on X-ray related imaging techniques, 
Stephan Jericho and his colleagues for their patience, Gustaaf Hallegraeff who 
shared his long-standing experience of Electron Microscopy techniques applied 
to planktonic species, and finally my students who took some of the burden of 
collecting information and writing. 

This book is not an encyclopaedia of all imaging techniques applied in marine 
biology. We made a choice and two main factors influenced our choices: three- 
dimension and promises. 

Eirst, water is always a three-dimensional medium that governs marine organ- 
isms: their shapes and forms but also, surely, their physiology. And so there is 
a crucial need to obtain three-dimensional views as this opens ways to deeper 
approaches: taxonomy, cell biology and in vivo biochemistry, fields well estab- 
lished in the medical world. Secondly, because there are so many imaging 
techniques out there, we had to make choices for this first edition, so we decided to 
drop the well-established techniques: light microscopy, cytometry . . . and promote 
less appreciated techniques or emerging ones that often give us a better three- 
dimensional view of the creatures of the oceans (Optical Projection Tomography, 
Light Sheet Microscopy etc.) and we made a big effort to make them available. 



We believe that these techniques can be highly beneficial to the marine biology 
community. 

However, we provide some basic skills to enable the reader to photograph larger 
organisms (Chapter IX) and a historical perspective of ocean imagery (Chapter I) 
as a reminder of where we are coming from to enlighten the path of where we 

go... 

This book is an introduction to marine biology imaging methods suitable not only 
for students wishing to pursue marine biology but also for established researchers 
eager to extend their knowledge of imaging methods that may improve their current 
or future research. 

This first book, specifically-designed for the marine biology field, is work in 
progress. We hope you will find it informative and your feedback is welcome to 
improve our common knowledge. 



Swim long and image well! 
Emmanuel G. Reynaud 



KahiKai 



Kahi Kai (‘one sea’ in Hawaiian) is a project born in Hawai’i. Kahi Kai stands for a unique 
ocean representing all oceans of our planet, which contain an amazing and mostly unknown 
biodiversity. The mission of Kahi Kai is to raise awareness about our ocean legacy and to appeal to 
our shared responsibility to preserve the fascinating, mysterious, and highly endangered marine 
world (www.kahikai.org). 



Some of the sea creature portraits featured as chapter openers were taken during the Tara Oceans 
expedition and are currently part of an itinerant exhibition that can be seen in 2013/2014 in 
French cultural centers and embassies throughout the world. 





(1) Noctilucasp. (Sea Sparkle) - Dinoflagellate, {2) - Distichopora sp. (Lace Coral) - Cnidarian, {3) Sabel - 
lastarte sp. (Sabellid worm)- Annelid, (4) Acropora sp. (Staghorn coral) - Cnidarian, (5) - Mastigias sp. 
(Lagoon jellyfish) - Cnidarian, (6) Chaetodon trifascialis (Chevron butterflyfish) - Vertebrate, (7) Physalia 
physalis (Portugese man o' war) - Cnidarian, (8) Myrianida sp. - Annelid, (9) Pseudoceros dimidiatus 
(Divided flatworm) - Platyhelminth, (10) Canthigaster papua (Papuan toby) - Vertebrate. 



Photo credits: Kahi Kai - (1) Noan le Bescot, (2, 5, 6. 7, 9, 10) Eric Rottinger, (3, 4, 8) Aldine Amiel, (1, 2 , 4, 6, 10) Tara Oceans. 
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1 

Under the Eye of Neptune: An Historical Perspective of Marine 
Creature Imagery 

Emmanuel C. Reynaud 



Abstract 

The ocean may have been filled by meteorites bringing water to Earth but its 
creatures were drawn by men. Sea shells were eaten in caves, dolphins drawn 
on cave walls. For a long time, though, its surface was the only visible thing, 
a glittering mirror for the human soul where Gods and Monsters fought over 
submerged empires, pulling down Atlantis and raising volcanic spires with 
thunder and smoke. Once the smoke had cleared and man learned to fish 
then dive, the marvels of the sea appeared and glittered even thousands of 
meters deep, revealing a magnificent world beyond our imagination. From 
the jellyfish’s ballet to intricate glass houses ofradiolarians, this introductory 
chapter presents a concise history of how man images the invisible oceanic 
world. Copyright © 2013 John Wiley & Sons, Ltd. 



1.1 

Introduction 

Black and dark was the Universe. Then, the Big Bang lit everything, making energy 
fiow, and amongst the billions of particles thrown across space was our planet, a 
tiny ball of orange fire. No blue, no green, no life. With time and changes, perhaps 
meteorite showers, water appeared on Earth and built up the marine world, or at 
least a place like an ocean, 4.6 billion years ago. 

Many theories have been proposed, discussed, discarded, and buried about the 
origin of life; often the ocean or marine environment is a component of the magical 
mix that gave rise to the chance for life on our planet. Cells appeared, close to deep 
sea volcanoes, or perhaps it was in rock pools; in a shallow bay stromatolites used 
the blazing sun’s energy to create food sources and deposits that mark the origin 
and began the continents. Life expanded and thrived in these early oceans but there 
was no one to record the creatures that inhabited it. Only bones and stones give us 
a glimpse of these creatures and for many millions of years there were no eyes to 
see them, and no arm and digits to draw them or to build cameras to picture this 
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massive world up to 12 km deep and spanning 70% of our Earth, from the frozen 
lands of Antartica to blooming coral reefs. 

Oceans covered the globe and currents carried organisms, continents emerged, 
and shades of green would have been seen from outer space. Blue was no longer 
the only color: brown, green, gray, and shades of sand were turning our world into 
a magnificent rainbow. Fishes grew limbs and left this ever moving world packed 
with predators, and along this line of evolution came the mammalians. They took 
over the world and even the oceans with the biggest of all those left alive: the 
whales. 

They developed and, eventually, stood up to look at the world around them, 
trying to understand it. They started to draw and we can witness dolphins jumping 
out of the water (Figure 1.1) painted on rock faces. These are the first known 
images of an ocean creature painted by men, probably 30 to 10 000 years ago. They 
imaged marine life on rock faces with natural pigments, a beautiful first attempt to 
approach the Great Blue Beyond. 

One of the most magnificent examples is L’Abri Poisson located in the valley of 
the Gorge d’Enfer, on the right bank of the Vezere River near Les Eyzies-de-Tayac. 
It was discovered in 1892 by Paul Girod, and dates from the Aurignacian (about 
47,000 to 27,000 years). This salmon, 1.05 m long, is etched and carved in low relief 
on the ceiling of the vault, enhanced with red pigment. And it has been carved with 
such accuracy that is attitude can be seen as a characteristic of a male exhausted by 
spawning. This is one of the only ten Paleolithic cave art fish known. 

Other examples of marine animals can be found in the Sierra de Guadalupe cave 
(Mulege in Baja Galifornia, Mexico), Gueva de la Pileta cave (province of Malaga 
(Spain) or Misool (West Papua, Indonesia (Figure 1.1)). 

Those fish drawings receive their origins from human’s desire to feed, clothe, 
and equip themselves with useful implements. According to Michael Barton, a 




Figure 1.1 Prehistoric rock art, Misool, 
West Papua, Indonesia. In the Misool area 
there are several sites decorated with exten- 
sive rock art or paintings. The paintings are 
thought to be between 3000 and 5000 years 
old and feature dolphins, fishes, whales. 



dugong, hands, abstract figures, and what 
may be representations of “Matuto," a half- 
man, half-lizard hero, or god. The sites are 
on sections of limestone cliff © Matthew 
Oldfield 2009. 
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prominent ichthyologist and professor at Centre College, “the earliest ichthyologists 
were hunters and gatherers who had learned how to obtain the most useful fish, 
where to obtain them in abundance and at what times they might be the most 
available”. 

Fish were not the only marine organisms used by the first humans of the Upper 
Palaeolithic but never painted; molluscs or rather their remains: shells. They 
were certainly uses as food source, raw material, ornaments, containers or even 
currency. 



1.2 

Ancient Uses of the Oceans 

For many years, the marine world was not only a source of food (fishes, shellfishes) 
but also, and mainly, a surface to move upon, a liquid pathway to new territories. 
But this dangerous blue world was deep and dark, feared as the house of Gods and 
Monsters able to swallow boats and seafarers in a single gulp. 

1 . 2.1 

Seafarers 

Knowing your environment and mastering it was a major leap in human activity. 
Early seafaring activity have been recorded in Sicily and Cyprus and dated to the 
Upper Palaeolithic period (30,000 to 11,000 years ago) but the Phoenicians are the 
most established sea-faring community (1,200BC) using the stars for their oceanic 
voyages. But Europe and the Mediterranean sea were not the birth place of human 
maritime activity. 

About 30 000 years ago, human cultures along the western coastline of the Pacific 
Ocean - in the area between what is now Australia and China - started to migrate 
eastward across the great expanse of the Pacific Ocean. We are not sure exactly why 
the migrations started, but tribal wars, disease epidemics, the search for food, or 
natural disasters, such as large volcanic eruptions and earthquakes, may have been 
factors. 

Over about 25 000 years, these people, called the Polynesians, eventually colonized 
the islands of the south and western Pacific, from New Guinea in the west to Fiji, 
and Samoa in the middle. They then moved onward to Tahiti and, finally, to Easter 
Island in the eastern south Pacific. How did the Polynesians manage to travel across 
thousands of miles of ocean without compasses, sextants, clocks, or other tools of 
modern navigation? Their migration was truly one of the great achievements of 
early seafaring cultures, and it marks the start of oceanographic observations by 
people who lived in harmony with the ocean. The Polynesians were very observant. 
They noted the directions that waves came from and how they affected or rocked 
their canoes. They had a keen sense of ocean currents and variations in bird and 
sea life in different places in the Pacific. They also were among the first people to 
use astronomical observations of the stars to help them navigate across the ocean. 
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They made the earliest form of navigational or oceanographic map, called stick 
charts. These were made of pieces of bamboo or other wood that were tied together. 
The locations of islands were often marked with shells or knots, and curved pieces 
of wood represented the bending of ocean waves around the islands and the way 
waves rocked their canoes. Polynesians handed down their lore of the sea in both 
the oral and stick-map traditions. 

We can suppose that they also recorded their world and discovery and a few pet- 
roglyphs of fishes and canoes can be found on Nuku Hiva, Marquesas Archipelago, 
France. 

1 . 2.2 

The Mediterranean Sea: the cradle of marine biology 

The references to the sea and its mysteries are myriad in Greek civilization, in 
particular in The Iliad and The Odyssey. However, in those two books about ancient 
history the sea is seen as the territory of Gods or as a means of moving from one 
point to the next and as providing food. Early Mediterranean civilizations, including 
the Greeks, have passed down many myths that include gods and goddesses who 
ruled over nature, such as Poseidon and his son Triton. Many Mediterranean 
legends, such as Jason and the Argonauts, also involved adventures on large and 
dangerous seas. 

About 2900 years ago, the Greeks began to venture outside the Mediterranean, 
past the Straits of Gibraltar at the western end of the Mediterranean Sea. This 
narrow channel separates Europe from Africa, and the Mediterranean from the 
Atlantic Ocean. Just outside of the Straits of Gibraltar, early Greek sailors noticed 
a strong current running from north to south. Because the sailors had only seen 
currents in rivers, they thought that this great body of water on the other side of 
the Straits was an extremely big river. The Greek word for river was okeano, which 
is the root of our word for ocean. 

1 .2.2.1 Aristotle and Pliny the Elder, the Founding Fathers 

Birth of Natural History Observation With the Greek world came the Academy, a 
world of knowledge, teaching, and exchange. A pupil of Plato, Aristotle, was the 
first to make specific comments on sea life. He had collected and referenced a vast 
number of species, including echinoderms, molluscs, and fishes, and he wrote the 
founding book of biology: “Animal stories.” 

In 344 BG, he went to the neighboring island of Lesbos, at Mytilene, to visit 
one of his colleagues from the Academy, the philosopher and naturalist - another 
student of Plato-Theophrastus (371-287 BG). They studied together the botany 
and the zoology of the island and also the creatures from the sea, or Thalassa, 
around it. Aristotle found on this island his second school, for almost two years. In 
343 BG, he went back to Macedonia, invited by King Philippe II (359-336 BG), to 
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become the preceptor of his son, the future Alexander the Great (336-323 BC). He 
was then appointed head of the Royal Academy of Macedonia. 

In 346 BC, Aristotle went to Atarneus with Xenocrates (ca. 396-314) and 
Theophrastus (ca. 372-288) and moved to the little harbor of Assos (the Turkish 
village of Berhamkale). Here he pursued further his biological research and started 
to observe marine life. He was the first to recognize that cetaceans where mammals 
and that those marine vertebrates where oviparous or viviparous. He described 
many forms of marine life and deduced gill function. He was the first zoologist 
and also the first to establish a classification of life forms. He mentioned nearly 
400 species. He divided them in two groups; the ones that have blood (Enhoema), 
more precisely red blood, the vertebrates, and the ones without: the invertebrates 
(Anhoema). Amongst the marine organisms described by Aristotle we find fishes 
and cetaceans and also the malacostraca (crustacean, crabs, etc.) as well as the 
Ostreioda (oysters) and the Mollusca: squids and octopus. The texts that have 
survived about biology, anatomy, and physiology seem to indicate that Aristotle 
performed dissections and probably drew them. But we have no record of the way 
zoology and marine biology was taught in the famous school of Alexandria during 
the Ptolemaic dynasty where the pupils of Aristotle further pursued his work. 

Interestingly, he also discussed tides and thought that there were due to the 
compression of lower sea water by surface waters. This error was later corrected 
by the Roman Pliny the Elder (v 23-79 BC), another father of zoology and marine 
biology, in his “Natural Stories.” He described these regular movements of the sea 
water (twice a day) as being caused by the Sun and the Moon. He observed two 
tides a day and, to quote, “In between two moon risings, the sea raises twice and 
‘gone’ twice in a 24 hour period” and he pointed out that the tides where never at 
the same time the following day. 

As he was the first to pass on his observations to us, Aristotle is considered to 
be the father of marine biology and zoology, even though he did not leave any 
drawings, only description of his observations. 

During the Roman Empire we only find zoological observations from Pliny, 
Solinus, and the agronomical writers. However, Pliny the Elder was mainly a 
“collector” of stories that he bound together in his “Natural Stories;” he does not 
provide much new knowledge. The only images of ocean creatures passed on to 
us are large fishes, dolphins in mosaics in the middle of Monsters, Gods, and 
Mermaids. 

1.2. 2. 2 Understanding the Oceans 

Early Science and Technology The oceans were a dangerous place to be and the 
lack of a reference point on this ever changing blue surface made it difficult to 
navigate. The Egyptians and the Phoenicians mainly used coastal navigation to 
conquer the Mediterranean Sea. Einally, the Greeks extended the range of their 
journey towards Gibraltar and in 325 BG Pytheas sailed from Greece to Iceland 
and following the North Star he observed latitude, correcting Aristotle. He also 
proposed that the tides were caused by the Moon. His followers, like Eratosthanes, 
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calculated the circumference of the Earth. Aristarchus independently calculated 
the diameter of the Earth from observations of the elevation of the Sun at noon 
at two locations along a north-south line between Greece and Egypt. Those first 
invisible reference lines using the stars made it possible to draw maps and Ptolemy, 
in 127-151 AD, produced the first world atlas, including the Pacific and Indian 
Oceans; he also showed the world as a globe. At the other side of the planet, the 
Polynesians developed advanced maps or stick maps that display key islands but 
mainly ocean currents that helped them to move great distances. 

finally, Hellenistic Greece developed coastal markers and lighthouses as nav- 
igational aids, one of which, the Pharos (lighthouse) of Alexandria, was one of 
the Seven Wonders of the Ancient World. They also probably established ways 
of accurately measuring time so as to be able to precisely determine latitude. 
However, such discoveries where only ways of using the surface of the ocean - 
the life striving in the ocean lay hidden under the waves, drifting with the Gods: 
Oceanus, Tethys, the nymph Eurynome, and the giant Ophion. 



1.3 

From Neptune to Animalcules 

Poseidon and his daughter Gharybidis were replaced by Neptune as the Roman 
Empire conquered the world, including the European coast lines. The desire to 
explore and to rule pushed ahead mainly on roads and oceans. Then there came 
trade and legends of gold and spices so far away by foot that European explorers 
turned to the sea to find faster trade routes to cities in Asia and Europe. Prince 
Henry the Navigator of Portugal (1420) recognized the oceans’ importance to trade 
and commerce and he established the first school of navigation. 



1.3.1 

Age of European Discovery and Exploration 

The fifteenth century saw Bartholomew Dias (1486) and then Vasco de Gama (1498) 
sailed around Africa and into the Indian Ocean while Golumbus sailed the Atlantic 
and (re)discovered the Americas, becoming the first European to sail westward 
across the Atlantic Ocean and return home. Ponce de Leon “discovered” (for the 
European) the Pacific Ocean, finally, the ultimate goal of circumnavigating the 
Earth was achieved by the Portuguese Eerdinand Magellan (1520) and later by Sir 
Erancis Drake (1580). 

It was not until the sixteenth century that zoology become a major subject of 
research and one of the earliest researchers was probably Guillaume Rondelet, 
known also as Rondeletus, in Montpellier who also, like Aristotle, performed 
dissection and explored not only fishes but also sea urchins and crabs. In 1556, he 
published “The Complete Story offish.” 
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1.3.2 

Voyages of Exploration and finally Science 

In the early 1700s, several European countries sought to expand their empires 
and discover new lands for raw materials, colonies or trade, and for spices from 
the East Indies, which they believed would help cure the Plague. They launched 
expeditions to survey faraway lands across the Atlantic, Pacific, and Indian Oceans, 
and in doing so also explored the Arctic and Antarctic Oceans. But only the lands 
were concerned and the oceans were merely a roadway to move from one land to 
another to claim them for their king, queen and other empires. The only sailors 
concerned about the sea creatures were the whalers. 

But in 1768, James Cook, on board the HMS Endeavour is sent to Tahiti 
on a scientific mission: to observe a rare eclipse caused by the interposition of 
Venus between Earth and the Sun. Passed through Cape Horn; he made his 
astronomical observations in June 1769. The British captain then took possession 
of the archipelago which he gives the name of the Society Islands, in honor of the 
Geographical Society of London which send him to Tahiti. Over 10 years, Cook 
led three world-encircling expeditions and mapped many countries, including 
Australia, New Zealand, and the Hawaiian Islands. He was an expert seaman, 
navigator, and scientist who made keen observations wherever he went. 

James Cook was not only an explorer. His travels have marked the end of 
the Great Discoveries and have allowed significant progress in the knowledge of 
the Pacific Ocean, with the hydrographic surveys and thoroughness of this great 
observations and reports. His expeditions were a huge success in Europe, which 
became fascinated with the stories of Cook, published along with remarkable maps 
and drawings. Louis XVI, during the Eranco-English war, ordered to befriend him. 
The Anglo-Erench competition, hosted by Bougainville and La Perouse, facilitated 
the expansion of geographical knowledge of the Pacific. James Cook began the era 
of scientific exploration. 

1.3.3 

A Glimpse at the Invisible 

The eighteenth century was not only a time of expedition but also of a major 
breakthrough for the future of biology. The magnifying power of segments of glass 
spheres was known to the Assyrians before the time of Christ; during the second 
century AD, Claudius Ptolemy, an astronomer, mathematician, and geographer 
at Alexandria, wrote a treatise on optics in which he discussed the phenomena 
of magnification and refraction as related to such lenses and to glass spheres 
filled with water. Despite this knowledge, however, glass lenses were not used 
extensively until around 1300, when an anonymous person invented spectacles for 
the improvement of vision. This invention aroused curiosity concerning the ability 
of lenses to magnify. In Holland in 1673, Anton Van Leeuwenhoek prepared quality 
lenses so that he could better examine garments. However, his insatiable curiosity 
led him to built one of the first microscope and use these lenses to examine many 
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Figure 1.2 Antonie van Leeuwenhoek “Animalcules.” The first pictures ever drawn of 
planktonic species defined by Antonie van Leeuwenhoek as “animalcules” or very tiny 
animals, which will become dilates, algae, Paramecium, and so on. 

biological items, including pond water. What he discovered in the water was a 
microscopic world of life. He described moving “animalcules,” later described by 
Christiaan Huygens in 1678 as paramecium (Figure 1.2). He described rotifers, 
Euglena and Spirogyra and paved the way to the use of microscope in biology and 
the extensive description of the invisible life in each drop of fresh or sea water. 



1.4 

The Birth of Oceanography (The Nineteenth Century) 

1.4.1 

Drawing the Jellyfish 

Nicolas Baudin in the Geographe and Emmanuel Hamelin on the Naturalist 
left Le Havre on October 19, 1800 for what will be called the Baudin expedition 
(1800-1804). Francois Peron (1775-1810), a French naturalist and physicist, aboard 
the Geographer, alongside the artist Charles-Alexandre Lesueur (1778-1846). Only 
a few scientists and crew members survive dysentery, fevers, deprivation and 
navigation in the southern storms. The journey ends at Port Jackson the 20th of 
June 1802. But before, the Geographer had to leave hurriedly King Island to escape a 
storm, and left Peron and Lesueur behind. They will be collected twelve days later by 
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an English ship. The return of the expedition, March 25, 1804 in Lorient, takes place 
in the utmost discretion, politics and rivalries in Europe monopolizing Napoleon. 

But the expedition brought a hundred unknown live animals to Europe (presented 
to the Empress Josephine) and nearly 100,000 samples. Of these, more than 2,500 
new species were described for the first time, Peron included in a rational and 
logical classification of high modernity. Also, Charles-Alexandre Lesueur realized 
the most vivid paintings based on live animals of jellyfishes, siphonophores and 
other fishes. Those were the first accurate depictions of those nearly invisible 
species, decades before Eorbes, Agassiz or Haeckel books on the subject but the 
Napoleon wars and the political situation obscured their beauty and the expedition 
reports were greatly ignored. 

1.4.2 

The H.M.S. Challenger Expedition 

The nineteenth century was an important time for the development of marine 
biology. However, the oceans were still mainly treated as a motorway where the 
priority was to establish safer and faster routes for good transports. The only 
experienced sailors exploring unknown territories for economical reason were the 
whalers. Those “biologists” by needs observed the whale behaviors, charted new 
routes and brought back astonishing legends such as the depth to which those 
animals could dive far below the scientifically approved limits of life in the deep 
oceans. Those observations and their books were used by Americans and well as 
British Admiralty to improve their charts. For example, Matthew Fontaine Maury, 
(1806-1873) the American father of modern oceanography, compiled analyses of 
wind and current logs, and published maps of wind and current patterns. He wrote 
the first oceanography textbook in 1855, “The Physical Geography of the Sea”. 

On the other side of the Atlantic, the Voyage of HMS Beagle (1831-1836) with 
Charles Darwin led to the publication of “The Origin of Species” but brought little 
to the history of marine biology apart from the theory of coral reefs formation. 

In 1852, the United States Navy Department decide to conduct a naval and 
commercial survey of those parts of Bering Straits, the North Pacific Ocean and 
the China Seas frequented by American whaling ships and by trading vessels 
in their routes between the United States and China. This was an economical 
expedition but Captain Cadwallader Ringgold was placed in command of the 
expedition, which departed from Norfolk, Virginia, in June 1853. And William 
Stimpson was appointed Chief Zoologist alongside naturalists such as Alfred H. 
Ames, Assistant Naturalist, and Charles Wright, Botanist. From 1853 tol856, they 
collected an extensive natural history collections were made. For nine years after the 
completion of the expedition, Stimpson utilized the facilities of the Smithsonian 
Institution while preparing a report based on the collections gathered during the 
expedition. While at the Smithsonian, he also helped classify and name specimens 
which the Smithsonian had been collecting. This great collection, including all the 
manuscripts being prepared by Stimpson, was destroyed in the Chicago fire of 
1871, an event from which Stimpson never fully recovered. 
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Those expedition reports were largely ignored by the European and mainly 
British scientists. Their main concern in oceanography was the “Azoic theory”. 
Edward Forbes, a distinguished British naturalist sailed with the British Survey 
on the HMS Beacon, charting the waters of the Mediterranean and Aegean Sea, 
where he was able to dredge for samples from the deepest waters examined to date 
based on European records ignoring American dredging n the North Pacific and 
the whaler tales. After these explorations, he developed his theory that below 300 
fathoms (~600 meters) the mysterious dark depths of the seas was a lifeless area 
and he named this the Azoic (lifeless) zone. His theory was accepted by the public 
but not by his colleagues such as Charles Wyville-Thomson who followed him in 
the Edinburgh Chair of Zoology. 

C. Wyville Thomson and W.B. Carpenter started large oceanographic cruises 
(HMS Lightning and HMS Porcupine) and started to falsify the “Azoic theory” 
bringing animals from below the Forbes 300 fathoms line with their dredges. These 
discoveries persuaded the British government to launch a worldwide expedition to 
explore the ocean depths not only because they wish to disprove Forbes theory but 
also because another important economical endeavour was a stake: the telegraph. 

The Challenger Expedition (1872-1876) - sponsored by the Royal Society and 
Royal Navy - left Portsmouth, England, just before Christmas 1872. The ship had 
many different types of samplers to grab rocks or mud from the ocean floor, and 
nets to capture animals from different levels in the ocean. It was the first expedition 
organized specifically to gather data on a wide range of ocean features, including 
ocean temperatures, seawater chemistry, currents, marine life, and the geology 
of the seafloor. The H.M.S. Challenger also had different winches - mechanical 
engines used to lower and hoist sounding lines to measure how deep the ocean 
was. At each sampling station, the crew lowered trawls, nets, and other samplers 
to different depths, from the surface to the seafloor, and then pulled them back on 
board loaded with animals or rocks. This British Navy corvette (a small warship), 
was converted into the first dedicated oceanographic ship with its own laboratories, 
microscopes, and other scientific equipment onboard (Figure 1.3). The expedition 
was led by British naturalist John Murray and Scottish naturalist Charles Wyville 
Thompson. 

H.M.S. Challenger first traveled south from England to the South Atlantic, and 
then around the Cape of Good Hope at the southern tip of Africa (Table 1.1). It 
then headed across the wide and very rough seas of the southern Indian Ocean, 
crossing the Antarctic Circle, and then went on to Australia and New Zealand. 
The expedition then headed north to the Hawaiian Islands, and then south again 
around Cape Horn, at the southern tip of South America where the Pacific and 
Atlantic Oceans meet. After more exploration in the Atlantic, they returned to 
England in May of 1876. 

It visited all of the world oceans, except the Arctic, covering 127,000 km. 
Scientists on board studied the physical conditions of the deep ocean, the chemical 
composition of seawater, the physical and chemical characteristics of the seafloor 
deposits, and the distribution of organic life at all water depths. As part of the 
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Figure 1.3 H.M.S. Challenger biological laboratory -This is a drawing of the first embarked 
biological laboratory to dissect, image, and analyze freshly collected planktonic and marine 
species - it would take 30 years to fully record and publish the results. 

Table 1.1 Some facts about the H.M.S. Challenger Expedition. 



Expedition crew 
Scientists 

Duration of expedition 
Distance sailed 
Number of sampling stations 
Number of depth soundings made 
Number of dredges taken 

Number of new species of animals and plants discovered 



243 

6 

4 years 

127000 km (68 890 miles) 
362 
492 
133 
4700 



cruise, scientists measured the water depth of the Marianas Trench (8185 m); some 
4717 new marine species were netted and classified. 

The expedition also tested and disproved Haeckel’s Bathybius theory (primordial 
slime) (Figure 1.4). Even though, Ernst Haeckel worked extensively on the samples 
brought back by the Challenger Expedition and will draw them to great length show- 
casing to the world new planktonic unicellular species: radiolarian, acantharian . . . 
Those marvelous illustrations and drawing made by Ernst Haeckel were used by 
Leopold and his son Rudolf Blaschka to create the unique marine glass invertebrate 
models displayed across Europe in Museums and used in many universities to 
teach marine biology and influence the new generation of biologists and micro- 
scopists (Figure 1.4). The later publication of his book “Art Forms of Nature” will 
influence artists of the Art Nouveau movement introducing the planktonic life 
forms in the general public. Rene Binet, a French painter and architect, studied 
extensively the works of Ernst Haeckel and his study of Haeckel’s lithographs of 
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Figure 1.4 Blaschka glass jellyfish - Leopold and Rudolf Blaschka designed a unique 
way to display in a full three-dimensional manner marine invertebrate such as jellyfishes, 
ctenophores, and microscopical planktonic organism (radiolarians). Courtesy of Francis 
Latreille. 

radiolaria culminated in the design for his Monumental Gate, located on the Place 
de la Concorde, at the Eastern entrance of the World Fair held in Paris in 1900. 
The design was heavily based on one radiolaria illustration alone - the Cyrtoidea 
Pterocanium trilobum. 

Most of all, the Challenger Expedition encouraged other countries to take an 
interest in the oceans and to mount their own expeditions such as the ones led 
by Prince Albert I of Monaco (Expeditions Princesse Alice I and II, Hirondelle I 
and II), who founded the Oceanographic Institute, which was later led by Jacques 
Cousteau. Prince Albert of Monaco led several expeditions and also developed an 
extensive knowledge of currents and collected maps. His institute knowledge of 
currents proved valuable during World War I as it allows military officials to predict 
how explosive mines would drift in the ocean and where they would land as in the 
twentieth century the oceans turn into battlefields. 

The German Empire was not left out of the Ocean Race for Knowledge and led 
two important expeditions: 1889 - National Expedition and 1898-1899 - Valdivia 
expedition. 

1.4.3 

Stations and Institutions 

But one major step forward was the establishment of stations and institutions that 
will foster marine biology imagery and progress. 

The College de France founded the first ever marine stations in Concarneau 
in 1859 which later extended to Roscoff on the northwest of France. Originally 
established for the cultivation of marine species, such as Dover sole, because of 
its location near marine estuaries with a variety of marine life, it evolves towards 
research on molecular biology, biochemistry, and environmental studies. 

A few years late, the Stazione Zoological was founded in 1872 by the German 
zoologist Anton Dohrn (1840-1909), a former student of Ernst Haeckel. Dohrn 
thought of the Napoli institute as a pilot centre of a worldwide network of research 
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facilities for studies in marine biology. He envisioned a railroad system where 
scientists could stop, collect material, make observations and conduct experiments, 
before moving to the next ‘station’: therefore the name “Stazione (=Station)”. 

At Woods Hole (USA) in 1888, the Marine Biological Laboratory (MBL) was 
established by Alpheus Hyatt, a student of Harvard naturalist Louis Agassiz who 
had established the first seaside school of natural history on an island near Woods 
Hole. MBL was designed as a summer program for the study of the biology of 
marine life for the purpose of basic research and education. The Woods Hole 
Oceanographic Institute was created in 1930 in response to the National Academy 
of Science’s call for “the share of the United States of America in a worldwide 
program of oceanographic research”. An independent biological laboratory was 
established in San Diego in 1903 by University of California professor Dr. William 
E. Ritter, which became part of the University of California in 1912 and was named 
the Scripps Institution of Oceanography after its benefactors. The MBL was also 
the first oceanographic institution to build the first oceanographic research vessel: 
“Atlantis" specifically for interdisciplinary research in marine biology, marine 
geology and physical oceanography. From 1931 to 1966, the “A- boat” made 299 
cruises and covered 700,000 miles, doing all types of ocean science. 

Interestingly the same year: 1888, a marine biological laboratory was established 
in Plymouth, England. This laboratory was established by the Marine Biological 
Association of the United Kingdom. 

Finally, the Monaco’s Oceanographic Museum and Laboratory opened in 1906 
and was followed by an extensive network of stations across the Globe. 

1.5 

The Twentieth Century: Institutions and moving images 

The twentieth century was a century of wars and technological developments. 

One of them, the sonar gave images of the ocean sea floor based on acoustic 
measurement. The possibilities to observe the marine creatures in their habitat 
rather than in specimen jar even at great depth was another important step 
especially when using moving images or video. 

1 . 5.1 

New tools - new images: 

Technology brought the study of marine biology to new heights. In 1934 William 
Beebe (1877-1962) and Otis Barton descended 923 m/3,028 ft below the surface off 
the coast of Bermuda in a bathysphere designed and funded by Barton. This depth 
record was not broken until 1948 when Barton made a bathysphere dive to 1,372 
m/4,500 ft. During the interim, Beebe was able to observe deep sea life in its own 
environment rather than in a specimen jar. Although he was criticized for failing 
to publish results in professional journals, his vivid descriptions of the bathysphere 
dives in the books he published inspired some of today’s greatest oceanographers 
and marine biologists. 
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In 1960, a descent was made to 10,916 m/35,813 ft in the Challenger Deep of the 
Marianna trench - the deepest known point in the oceans, 10,924 m/35,838 ft deep 
at its maximum, near 11° 22’N 142° 36’E - about 200 miles southwest of Guam. 
The dive was made in the bathyscape “The Trieste” built by Auguste Piccard, his 
son Swiss explorer Jean Ernest-Jean Piccard and U.S. Navy Lieutenant Don Walsh. 
The descent took almost five hours and the two men spent barely twenty minutes 
on the ocean fioor before undertaking the 3 hour 15 minutes way up. 

Several technological developments allowed the marine biologists to record 
images in situ but also with better resolution (microscopes) and allowed the 
publications of pictures, photographs and videos. Remotely Operated Vehicle 
(ROV) and the development during the Second World War of scuba diving brought 
men further into the oceans as well as the establishment of ocean observatories, 
permanent stations connected to shore with optical cable. 

1.5.2 

jean Painleve 

Jean Painleve (1902-1989) was raised by his aunt, Mary, widow and sister of Paul 
Painleve, after his mother. Marguerite Petit de Villeneuve is carried by puerperal 
fever shortly after his birth. He was not the best at school and skipped classes to go 
to the Zoo where he helps the guardian to care for animals. He managed to attend 
the Lycee Louis Le Grand, Paris and began studying medicine in 1921. Two years 
later, he slams the door in the face of Professor Delbet after a disagreement over 
the treatment of hydrocephalus patients that he estimated a cruel treatment. He 
turned to biology and begins to attend the Station Biologique de Roscoff where he 
met Alexander Galder and the surrealist movement. He also started to experiment 
with cameras and from 1927 he spent all his time making documentaries whose 
subject is the world of marine animals, shrimp, octopus, sea urchins and other 
shellfish. Between 1928 and 1930, he shot a dozen films, including several for 
the general public: “The Octopus”, “The Urchin”, “The Hermit Grab”, “Grab and 
Shrimp”, “Gaprelles and Pantopodes” and “Hyas and Stenorinques”. Some of 
those films were presented in Paris and the press was very generous but also artists 
such as Eernand Leger or Marc Ghagall. One of his movies “The SeaHorse” was 
distributed by Pathe and established Jean Painleve as an artist but also the first 
scientific filmmaker and a movie maker of marine organisms. Painleve, through 
film, wanted to show the bustling life of the underwater world to the “unsuspecting 
public”. He realized more than 200 films and inspired not only artists but also 
scientists the ever expanding use of cinema and videos in marine biology. 

1.5.3 

The Writers and the Explorers 

The twentieth century was a time of media and communication that reached out 
and gave the unique opportunity to show to the public the oceans and its creatures 
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with books, novels than movies, TV series and online pages. Some people are 
well-known for their dedication to the marine world. 

Rachel Carson (1907-1964) was a scientist and writer who brought the wonders 
of the sea to people with her lyrical writings and observations about the sea. 
Although she was a biologist for the US Fish and Wildlife Service, she devoted her 
spare time to translating science into writings that would infect the reader with her 
sense of wonder and respect for nature. She published a book in 1941 titled “ Under 
the Sea-Wind.” These publications described the sea and the life within it from a 
scientist’s point of view, but in the words of a naturalist. In 1951, she published 
“The Sea Around Us” a prize-winning bestseller on the history of the sea. 

Dr. Robert Ballard (1942-), a deep-sea explorer, may be best known for finding 
the Titanic, and helping James Cameron, using technologies he helped to develop, 
including the Argo/Jason remotely operated vehicles and the technology that 
transmits video images from the deep sea. His earlier deep sea explorations led 
to the first discovery of hydrothermal vents during an exploration in a manned 
submersible of the Mid-Ocean Ridge. Ballard spent 30 years working on the use of 
manned submersibles. Ballard has devoted a great deal of time to furthering the 
field of deep sea exploration. He created a distance-learning program with more 
than one million students enrolled, taught by more than 30,000 science teachers 
worldwide. 

Jacques Cousteau (1910-1997) was determined to safely breathe compressed air 
underwater in order to lengthen dive times. His work with Emile Gagnan ultimately 
led to the invention of the regulator which releases compressed air to divers “on 
demand” (as opposed to a continuous flow). The combination of the Cousteau- 
Gagnan regulator with compressed air tanks allowed Gousteau the freedom to film 
underwater, and by 1950 he had produced the Academy Award winning “The Silent 
World.” By the 1970s he was bringing the underwater realm into millions of homes 
with his PBS series “Cousteau Odyssey.” Gousteau’s television documentaries won 
40 Emmy Awards. Like other oceanography pioneers, Gousteau was criticized for 
his lack of scientific credentials; however his legacy fostered a greater knowledge 
and understanding of the devastation caused by threats to ocean health such as 
pollution of marine resources and resource exploitation. 

Dr. Hans Hass (1919-2013), the Gousteau’s Austrian counterpart, also helped 
introduce the wonders of the underwater world to the public. Hass and his 
wife Lotte were both passionate about underwater exploration and protection of 
the marine environment, and together they produced numerous documentaries 
and wrote a variety of books on their underwater experiences. During his career 
as an underwater explorer, Hass also made significant contributions to diving 
technology. He invented one of the first underwater flash cameras and contributed 
to the development of the Drager oxygen rebreather which he and Lotte used in 
1942 to film “Men Amongst Sharks” and continued to use on diving expeditions 
aboard their research vessel “Xarifa” in the Red Sea and Garibbean. Hass is also 
known as one of the first humans to interact with a sperm whale underwater which 
helped him become a pioneer in the study of marine animal behavior. 
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1.5.4 

The Future 

Today, the possibilities for ocean exploration and imaging are nearly infinite. In 
addition to scuba diving, fast computers, remotely-operated vehicles (ROVs), deep 
sea submersibles, reinforced diving suits, and satellites, other technologies are also 
being developed. In spite of ongoing technological advances, it is estimated that 
only 5% of the oceans have been explored. Surprisingly, we know more about 
the moon than we do the ocean. Many new discoveries await us as we use new 
instruments and deep submergence vehicles to explore the “inner space” in the 
twenty-first century. 

In future, oceanographers want to go beyond learning about what is down 
there in the ocean to learning about what is going on down there. They want to 
observe oceans processes that change over days, weeks, seasons, years, or decades. 
However, it is difficult and expensive to send research ships back to the same 
site for repeat measurements. Sometimes rough seas and stormy weather make it 
impossible to send ships to certain parts of the oceans at certain times. 

Consequently, oceanographers are launching a new era of ocean exploration. 
They want to establish long-term ocean floor observatories with arrays of sensors 
and instruments that make continuous measurements of various ocean properties 
and events. Data from the observatories will be sent to shore-based laboratories via 
submerged fiber-optic cables or via cables linked to moored buoys that can transmit 
data via satellite. The data can then be made available via the Internet. 

Oceanographers will, in future, use different types of ROVs and AUVs that can 
“fly” in the oceans or along the seafloor, collecting measurements. The data can be 
downloaded when the AUVs surface, or when they dock at an underwater docking 
site. Oceanographers are also developing instrumented buoys moored thousands 
of miles from shore, and free-floating drifting instruments that can transmit data 
to scientists in their laboratories using satellites and the Internet. 



1.6 

Time Line of Ocean Imagery 

• 30,000 BC (Upper Paleolithic Period): Cave painting and carving of fishes and 
marine mammals in Europe and South East Asia 

• 30,000 BC —3,000 BC: Polynesians then Mediterranean civilizations sailed the 
oceans and master currents, star guided navigations and named the Great Salted 
River, okeanos. 

• 384 BC —322 BC: Aristotle, Greek philosopher and polymath wrote the first treaty 
of marine zoology. 

• 323 BG —32 BC: Hellenistic Period. Marine mammals such as dolphins and 
fishes are represented in mosaics, vases but also in sculpture as gods mounts. 
Greek sailors further extended Thalassa limits. 
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27 BC-AD 476: Roman Empire. Marine mammals such as dolphins and fishes 
are represented in mosaics, vases but also in sculpture as gods mounts. 
1507-1566: Gulielmus Rondelet wrote libri de piscibus marinis, whose works are 
almost entirely limited to Mediterranean and European fishes. 

1530: The first longitude is established but it will not be until 1737 and John 
Harrison that the first precise marine chronometer will appear. 

1632-1723: Antonie Philips van Leeuwenhoek: He described and draw for the 
first time "animalcules” found in a drop of water (rotifers, euglena, spirogyra). 
1728-1779: James Cook: The famous British explorer is well-known for his 
extensive voyages of discovery for the British Navy. He introduce the presence 
of scientists onboard expedition vessels who defined the modern day study of 
marine biology. 

1842: Darwin publishes The Structure and Distribution of Coral Reefs, in which 
he suggests that coral atolls are the final stage in the subsidence and erosion of 
volcanic islands. 

1843: Edward Forbes (1815-1854), after sailing on the HMS Beacon in the 
Mediterranean, develops the Azoic theory (no life deeper than 300 f= 1800 ft). 
This is the first marine scientific hypothesis! It started a 20-year debate on the 
presence of the lifeless (azoic) zone. 

1857: Charles Darwin (1809-1881) - Voyage of the HMS Beagle: circumnavigated 
the world; theory of coral reef subsidence (drilling in the Enewetak atoll partially 
confirmed the hypothesis); barnacle classification. 

1859: The first oceanographic station is established at Concarneau in France. 
1868-1869: Wyville Thomson dredges from HMS Lightning and Porcupine and 
discovers life as deep as 2400 fathoms, exploding forever Edward Forbes’ theory 
of a lifeless (azoic) zone below 300 fathoms. 

1872-1876: The Challenger Expedition circumnavigates the globe in the first 
great oceanographic expedition. Research is conducted on salinity, density, and 
temperature of sea water as well as ocean currents, sediment, and metrology. 
Hundreds of new species are discovered and underwater mountain chains 
documented. Modern oceanography is based on this research. The British 
Challenger expedition was led by Sir Wyville Thomson, who died before all of 
the results are compiled. Sir John Murray finished the great work, publishing 50 
volumes of the Challenger’s results and discoveries. 

1872: The Stazione Zoological was founded in by the German zoologist Anton 
Dohrn (1840-1909) 

1888: Marine Biological Laboratory (MBL) was established by Alpheus Hyatt, a 
student of Harvard naturalist Louis Agassiz at Woods Hole (USA). 

1888: The Marine Biological Association of the United Kingdom established the 
marine biological laboratory Plymouth, England. 

1890-1898: First Austrian -Hungarian deep sea expedition on board the ship 
SMS Pola led by Franz Steindachner in the eastern Mediterranean and the Red 
Sea. 
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• 1898-1899: First German deep sea expedition on board the ship SMS Valdivia 
led by Carl Chun; found many new species from depths greater than 4000 m 
(13000 ft) in the southern Atlantic Ocean. 

• 1906: Establishment of the Monaco’s Oceanographic Museum and Laboratory 

• 1930: Jean Painleve shot the first movies of marine organisms 

• 1930: William Beebe and Otis Barton are the first humans to reach the Deep Sea 
when diving in the so-called Bathysphere, made from steel. They reach a depth 
of 435 m (1430 ft), where they observed jellyfish and shrimp. 

• 1930: Establishment of the Woods Hole Oceanographic Institute and build the 
“Atlantis” the first oceanographic vessel dedicated specifically for interdisciplinary 
research in marine biology, marine geology and physical oceanography. 

• 1934: Edward Beebe is lowered in a tethered bathyscaph to a depth of 3028 ft, 
marking the advent of manned exploration of the sea. William Beebe dives in 
his bathysphere to a depth of 908 m on 14 August 1934. He is the first to observe 
firsthand life at great depth. 

• 1942: Hans Hass film and present the movie: “Men Amongst Sharks” 

• 1950: Jacques Cousteau produced the Academy Award winning “The Silent 
World.” 

’ 1960: Bathyscaph Trieste dives to what was believed to be the deepest point in 
the Mariana Trench: a depth of 10 915 m was observed. Since then, in 1998, a 
Japanese research vessel measured 10 938 m in the same area. The trench was 
first sounded by H.M.S. Challenger in 1875, and again by H.M.S. Challenger II in 
1951. 

• 1970: Jacques Cousteau brings the underwater realm into millions of homes with 
his PBS series “Cousteau Odyssey.” 



Further Reading 

Basic Texts 

Deacon, M. (1997) Scientists and the Sea 
1650-1900: A Study of Marine Science. 
Aldershot, UK, Brookfield, VT: Ashgate, 
2d edn, pp. 459 

McConnell, A. (1982) No Sea Too Deep: 

The History of Oceanographic Instruments 
(Bristol: A. Hilger), pp. 162. 

Mills, E.L. (1989) Biological Oceanography: 
An Early History, 1870-1960. (Ithaca, NY: 
Cornell University Press, pp. 378. 

Source Books 

Deacon, M.B. (ed.) (1978) Oceanography, 
Concepts and History. Benchmark Papers 
in Geology, Vol. 35, Stroudsburg, PA: 
Dowden, Hutchinson & Ross, pp. 394. 



Deacon, G.E.R. and Deacon, M.B. (eds) 
(1982) Modem Concepts of Oceanography, 
Benchmark Papers in Geology, Vol. 61, 
Stroudsburg, PA: Hutchinson & Ross, 
pp. 385. 

Idyll, C.P. (ed.) (1969) The Science of the Sea: 
A History of Oceanography, New York: T.Y. 
Crowell, pp. 280. 

Murray, J. (1885) in Report on the Scientific 
Results of the Voyage of H.M.S. Challenger 
During the Years 1873-1876, Narrative, 
vol. I First Part, London: John Murray, 
pp. xxxi-liv. 

Murray, J. (1912) in The Depths of the Ocean, 
J. Murray, J. Hjort (eds), London: Macmil- 
lan, pp. 1-21. 

Schott, W. (1987) Early German Oceano- 
graphic Institutions, Expeditions and 



Further Reading 



21 



Oceanographers, Hamburg: Deutsches 
Hydrographisches Institut, pp. 50. 

Thomasson, E.M. (ed.) (1981) Study of the 
Sea: The Development of Marine Research 
under the Auspices of the International 
Council for the Exploration of the Sea, 
Farnham: Fishing News Books, pp. 253. 

Vaughan, T.W., et al. (eds) (1937) Inter- 
national Aspects of Oceanography: 
Oceanographic Data and Provisions for 
Oceanographic Research, Washington, DC, 
National Academy of Sciences, pp. 225. 

Ships and Expeditions 

de Buen, R. (1934) Liste Chronologique des 
Croisieres Oceanographiques: Manuel 
pour les Observations Oceanographiques 
a la Mer-Commission Internationale pour 
I’Exploration Scientifique de la Mer Mediter- 
ranee 1, Venezia: C. Ferrari, pp. 73. 

Estok, R. and Boykin, R.E. (1976) A Union 
List of Oceanographic Expeditions Includ- 
ing Results of Some Major Cruise Reports, 
College Station: Texas A&M University 
Libraries, pp. 138. 

Guberlet, M.L. (1964) Explorers of the Sea: 
Famous Oceanographic Expeditions, New 
York: Ronald Press, pp. 226. 

Rice, A.L. (1986) British Oceanographic Ves- 
sels, 1800-1950, London: Ray Society, pp. 
193. 

Schlee, S. (1978) On Almost Any Wind: The 
Saga of the Oceanographic Research Vessel 
Atlantis, Ithaca, NY: Cornell University 
Press, pp. 301. 

Wust, G. (1964) The major deep-sea expedi- 
tions and research vessels, 1873-1960, a 
contribution to the history of oceanogra- 
phy, Prog. Oceanogr. 2, 1-52. 

Institutions 

D’Errico, F., Henshilwood, C., Lawson, G., 
Vanhaeren, M., Tillier, A.M., Soressi, M., 
Bresson, F., MaureiUe, B., Nowell, A., 
Lakarra, J., BacweU, L. and Julien, M. 



(2003) Archaeological Evidence for the 
Emergence of Language, Symbolism, and 
Musisc: An alternative Multidisciplinary 
Perspective, Journal of World Prehistory 17 
(1), 1-51. 

Limburg, P.R. (1979) Oceanographic Institu- 
tions: Science Studies the Sea, New York: 
Elsevier/Nelson Books, pp. 265. 

Metz, C.B. (ed.) (1985) The Naples Zoological 
Station and the Marine Biological Lab- 
oratory: One Hundred Years of Biology, 
Biological Bulletin, vol. 168 Supplement, 
The Marine Biological Laboratory, pp. 
1-207. 

Pinsell, M.L (1981) 150 Years of Service on 
the Seas: A Pictorial History of the U.S. 
Naval Oceanographic Office from 1830 to 
1980, vol. 1 (1830-1946), Washington, DC, 
Department of the Navy, Oceanographic 
Office, pp. 233. 

Raitt, H. and Moulton, B. (1967) Scripps 
Institution of Oceanography: First Fifty 
Years, San Diego, CA: Ward Ritchie Press, 
pp. 217. 

Revelle, R. (1980) in Oceanography: The Past 
(eds M. Sears and D. Merriman), New 
York, Heidelberg, Berlin: Springer-Verlag, 
pp. 10-24. 

Sahrhage, D. (1982) 80 Jahre Deutsche 

wissenschaftliche kommission fur Meeres- 
forschung. Meeresforschung 29, 197—207. 

Schott, W. (1987) Early German Oceano- 
graphic Institutions, Expeditions and 
Oceanographers, Hamburg: Deutsches 
Hydrographisches Institut, pp. 50. 

Shor, E.N. (1978) Scripps Institution of 
Oceanography: Probing the Oceans 1936 
to 1976, San Diego: Tofua Press, pp. 502. 

Southward, A.J. and Roberts, E.K. (1987) 

One hundred years of marine research at 
Plymouth. J. Mar. Biolog. Assoc. U.K. 67, 
465-506. 



23 



2 

New Solutions in Underwater Imaging and Vision Systems 

Francisco Bonin-Font, Antoni Burguera, and Cabriel Oliver 



Abstract 

Underwater vehicles, with autonomous capabilities, are becoming increas- 
ingly available to assist or replace divers and remotely guided vehicles used 
for ocean exploration or for the maintenance of submerged installations. 
Traditionally, autonomous underwater vehicles have used acoustic sensors 
to perceive the environment or to navigate. However, although images of 
marine objects can be distorted by the seawater, optical imaging systems can 
clearly benefit many underwater robots. The European project Trident aims 
to develop a new multipurpose autonomous underwater platform, usable for 
exploration and intervention and with diverse scientific, industrial, and rescue 
applications. One of the main tasks included in Trident is the incorporation 
of an imaging infrastructure to run some of the projected functionalities: sur- 
veying, scene recognition, object identification, and station keeping. When 
a vision system for an underwater vehicle has to be designed, some specific 
characteristics of the image formation in sub-sea conditions should be taken 
into account. This chapter first presents an extensive survey of components, 
techniques, and methods used to build underwater vision systems. In partic- 
ular the following are reviewed: characteristics of images formed underwater, 
some light source solutions, a list of relevant vision-based equipment and 
systems, as well as some methods for improving the image. Finally, a new 
imaging system developed in the context of the Trident project is described. 



2.1 

Introduction 

Thanks to the latest technological advances, the submarine world is more accessible 
for exploration, scientific research, and industrial activity. Sampling the oceans has 
traditionally been carried out from ships and using divers, but nowadays, unmanned 
underwater vehicles play a very important role in the sub-sea exploration. 

ROVs (remotely operated vehicles) have been used for several decades in 
many submarine tasks, but recently, AUVs (autonomous underwater vehicles) 
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have emerged as a suitable alternative to ROVs or divers, especially for highly 
repetitive tasks, in hazardous environments or for long missions. Currently AUVs 
are being used for various tasks, such as surveying, mine countermeasures, 
inspection, and maintenance of underwater infrastructures and wreck search and 
localization. Because they are untethered and self-powered, AUVs offer a significant 
independence from surface support ships and weather conditions. This can notably 
reduce the operational costs and the complexity of the required human and material 
resources, in comparison to operations conducted with tethered ROVs. 

Detailed analyses of underwater autonomous robots shows that range sensors, 
such as acoustics or laser, have been traditionally used. However, visual sensors 
are emerging as good alternatives or complements to range sensors because of 
the low cost of the cameras, the richness of the data provided, and the better 
spatial and temporal resolution. Obviously, the tjrpe of sensors used to perceive 
the environment determines the final capabilities of the robot. Water presents 
limitations to almost all types of physical waves, so the sensing equipment must 
be designed to overcome these problems. In particular, owing to the interaction 
between water and electromagnetic waves, sub-sea images can suffer from several 
disruptive effects that need to be taken into account during the light-gathering 
process: namely, light attenuation and scattering, refraction, non-uniform lighting 
and shadows, and color filtering. Additional problems are created by suspended 
particles or the abundance of marine life surrounding the target of interest. Some 
techniques specifically designed to overcome these problems include the use of 
specialized illumination systems, filters, ultra-sensitive and wide-spectrum cam- 
eras, or multi-camera configurations (Kocak et al., 2008). Other authors have shown 
how appropriately modeling the physical process of image formation, including 
information from the radiant source, the nature of the surface of scene objects or 
the imaging hardware, can improve the performance of some visual algorithms 
(Singh, Howland, and Pizarro, 2004; Negahdaripour, 1998; Ortiz and Oliver, 2010). 

Ultrasounds are not affected by underwater optical-imaging problems, but sonar 
provides information at a relatively slower rate and less resolution. Laser light 
is affected by all the problems associated with using light as listed above, and 
although various techniques can ameliorate some of these, laser-based systems are 
still expensive. 

Although the quality of images in subaquatic environments is strongly limited by 
the water and the illumination conditions, under certain situations optical cameras 
offer definite advantages over other sensors. Imaging platforms are not really 
appropriate in large scale deployments of AUVs or in open waters where it can be 
difficult to see the seabed, natural or man-made structures, or other reference points. 
However, for surveying or intervention applications, where the vehicle can navigate 
close to the bottom or it has to stay near the object that has to be manipulated, the 
use of cameras can certainly be a convenient option. Images taken at the pertinent 
distance to the target with a suitable illuminating infrastructure, can supply the 
robot with visual data of sufficient quality. Three examples of applications where 
cameras play an important role in the achievement of the desired task are: (i) visual 
mosaicking approaches attempt to overcome the difficulty of underwater visual 
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perception by aligning short range images of the sea bottom to compose a 2D or 
3D reconstruction of larger areas (Campos, Garcia, and Nicosevici, 2011), (ii) visual 
station keeping approaches are used to maintain the robot in a predefined pose in 
front of the object of interest, usually when a manipulation task is being performed 
(van der Zwaan, Bernardino, and Santos-Victor, 2002), and (iii) some underwater 
visual localization techniques that detect and track visual features from images 
of the sea bottom, computing, afterwards, spatial transformations to estimate the 
robot motion (Botelho, Drews, and Leivas, 2008). 

In this context, the European project Trident (2010-2013) has coped with 
the challenge of developing a system that combines the facilities of exploration, 
localization, and manipulation, and of performing complex tasks with minimal 
operator intervention. The generic mission proposed in the project consists of 
two phases: exploration and manipulation. Both main tasks are highly supported 
by a new visual infrastructure, developed by the Systems, Robotics, and Vision 
Group at the University of the Balearic Islands and designed to work with 
different underwater platforms. This visual system is dedicated to several functions, 
namely: seabed data collection for later mosaicking, object recognition, scene 
characterization and identification, visual docking to the support ship, and station 
keeping or odometry. The AUV Girona-500 (Ribas et at., 2012) is being used 
to support part of the experiments and can be endowed with different new 
manipulators to permit intervention capabilities. The Girona-500 vehicle was 
developed as part of a Spanish project and designed as a reconfigurable system 
formed by three torpedo-shaped hulls that house the equipment and computers 
that manage the different functions of the robot. 

This chapter now focuses on two main issues: 

1) A review of present underwater imaging systems, paying special attention to 
available cameras, to the distorting effects of seawater, to different possibilities 
for illuminating the environment, and to possible techniques to improve 
the quality of the images based on the use of polarized light. Moreover, 
a list of underwater installations and submarine vehicles equipped with 
imaging infrastructures, recently developed and commercially available has 
heen included as a record of previous applications. 

2) A description of the aforementioned new vision system, called Fugu. As will be 
described in later sections, Fugu consists of a watertight cylinder containing 
a computer, some additional electronic cards and several internal sensors. 

The system interacts with two stereo cameras. Stereo images are processed in 
the same system to provide online the information required to other parts of 
the robot. 



2.2 

Underwater Optical Image Formation 

When electromagnetic waves propagate in a sub-aquatic medium they interact 
with water molecules and with dissolved and particulate matter. As a consequence. 
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the distance this radiation travels in the underwater environment is dramatically 
reduced compared to air. A general exposition of this situation from a physical 
point of view can be found in Gordon (1994) and Mobley (1994). A detailed analysis 
regarding the underwater light propagation problem, focused on computer vision 
and its applications in robotics is presented in Ortiz (1998) and Horgan and Toal 
(2009), respectively. Among all the optical issues known, refraction, scattering, 
and absorption seem to play a more significant role in underwater optical image 
formation and computer vision and they are described below. 

Underwater cameras are housed in watertight enclosures including a depth rated 
lens. Before reaching the sensitive area of the camera, refraction causes the light 
rays coming from the scene to bend as they pass from water to glass and then from 
glass to air. Refraction modifies the apparent size and position of objects. This 
effect combined with imperfections of the housing system, including lens defects 
and misalignments, lead to nonlinear image distortion that must be compensated 
with a proper calibration process. 

When a photon hits a particle suspended in the water its original path is deflected. 
Depending on the angle the light ray is deviated, this phenomenon is known as 
forward scatter or backscatter. Forward scatter occurs when the angle of deflection is 
small, resulting in image blurring and contrast reduction. Backscatter occurs when 
part of the light sent by the source is reflected to the camera before reaching the 
object to be illuminated. Backscatter may cause bright points in the image, which 
are usually known as marine snow. However, the main problem with backscatter, 
also referred to as veiling light, is that it can highly reduce the image contrast, 
causing serious problems in underwater imaging systems. The general term for 
this problem, which also appears in other media than water, is path radiance. 
Figure 2.1 illustrates the effects of backscatter, forward scatter, and refraction. 

Both backscatter and forward scatter depend on the volume of illuminated water 
inside the camera’s field of view. In general, this means that the negative effects of 
scatter in underwater imaging increase with the distance from the camera to the 




Figure 2.1 Example of backscatter, forward scatter, and refraction. 
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object to be viewed. Moreover, as will be explained later, some veiling and snow 
effect reduction can be obtained by increasing the distance from the light source to 
the camera in an opportune way. 

Absorption causes the electromagnetic waves traversing water to be quickly atten- 
uated. Furthermore, the spectral components of light are differentially absorbed. 
Thus, in clear water long wavelength (red light) is lost first. In turbid waters, or in 
places with high concentration of plankton red light may be better transmitted than 
blue light. As a consequence, two problems arise that have important consequences 
for optical imaging and computer vision systems. First, the use of artificial lighting 
is needed in most cases and dramatically limits the distance at which objects are 
perceived. Second, the natural colors are distorted and the perception of the scene 
can be altered. 

The overall effects of absorption, scatter, and other phenomena not described 
here, such as fluorescence, produce an attenuation of light. The attenuation factor 
depends on the specific characteristics of the water, the dissolved components and 
the particles in suspension. A common measure of this effect is the attenuation 
length, which is the distance where the intensity of the light has dropped by a 
factor of 1/e. In other words, the attenuation length is the distance after which 
about 64% of the light intensity is lost. Visibility in water ranges between one 
and two attenuation lengths. This means, in general, ranges from 30 to 60 m in 
clear waters, 6-15 m in deep waters, and 1.5-6 m in coastal waters. As will be 
shown throughout, using the appropriate illumination and camera can significantly 
increase the visibility range. 



2.3 

Illumination Techniques 
2.3.1 

Illumination Sources 

Choosing the proper light sources for an efficient underwater visual system 
becomes an important issue because illumination conditions always determine 
certain imaging effects and results. Natural light sources, like daylight, can be used 
at low depths, but they completely attenuate before they reach significant depths. 
Artificial light is necessary at deeper environments, and since a single visible light 
source illuminates the scene producing a central bright spot surrounded by a poorly 
illuminated area, it is common to use different light sources strategically located, 
especially if they emit structured light. 

From the illumination type point of view, underwater imaging systems can be 
classified according to different concepts. In a first approximation, illumination 
systems can be roughly classified as active or passive. On the one hand, passive 
illumination systems image scenes illuminated by some kind of natural lighting 
source (sunlight or bioluminescence sources) or by some artificial source non 
specifically placed to illuminate that environment (light coming from nearby 
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stations, ships or whatever that is generating and/or consuming energy). Passive 
imaging is especially attractive for covering operations such as fish seeking, 
defense, or surveillance tasks. On the other hand, active illumination and structured 
lighting systems take advantage of artificially generated light with one or more 
sources strategically placed and configured. These systems offer substantial benefits 
for underwater imaging because the incident light can be either continuously 
emitted (standard visible light sources), collimated into narrow or wide beams, 
monochromatic (lasers), or flashed (strobe, pulsed lasers). These more sophisticated 
lighting systems typically allow imaging at greater ranges and/or higher depths 
than passive systems. The most significant of the advanced techniques that use the 
effects of light at specific frequencies to overcome harmful effects such as scatter, 
refraction, or absorption, among others, are: 

• Synchronous scanning systems: The illumination source emits a collimated light 
(rays are nearly parallel) with a minimal beam section. This causes minimum 
backscatter and it results in high contrast images. To compute target range, 
triangulation can be used (Zheng et al, 2009). 

• Light stripe range scanning {LSRS) \ A plane or sheet of light, typically generated 
by a laser diode, scans the environment or an object to be imaged to obtain its 3D 
reconstruction (Narasimhan and Nayar, 2005; Taylor and Kleeman, 2006). In the 
presence of scattering, the light sheet becomes visible and it makes the detection 
of the obstacle surface more complicated. This technique reduces backscatter 
and allows the recovery of 3D information by means of triangulation. 

• Photometric stereo (PS): Photometric stereo techniques are a good alternative 
to LSRS techniques when these take too long, for example, in dynamic environ- 
ments. In the absence of scattering, it is well known that three images obtained 
by illuminating the scene from three different but known directions are enough 
to reconstruct the surfaces of the different scene objects. A challenging problem 
arises when it is necessary to determine how many sources are needed to infer 
the scene features in the presence of scattering (Negahdaripour, Zhang, and 
Han, 2002; Narasimhan and Nayar, 2005). 

• Range gated systems: The source emits a pulse of light and the camera shutter 
waits for the time the light takes to propagate from the emitter to the target, 
scatter in the target surface, and back again to the camera. Only the light scattered 
by the target is received and considered for imaging. The main difficulty with 
these systems is that a very precise light gating is needed in the camera receptor. 
Obviously, the light pulse is much shorter than the total light propagation time. 
The cost of these systems is high but they considerably reduce the backscatter 
and augment the contrast (Han, Zhang, and Ge, 2009; Tan et al., 2006). 

2.3.2 

Selection of the Light Source Position 

When using conventional illumination systems, the amount of backscatter depends 
on the volume of water where the light field and the camera’s field of view intersect. 
Because of this, it is common to separate the light source as much as possible from 



2.3 Illumination Techniques | 29 

the camera in order to reduce the aforementioned volume of water. Figure 2.2a 
exemplifies the volume of water producing backscatter when the camera and 
the light source are close together, and Figure 2.2b shows the difference when 
increasing this distance. Jaffe (1990) emphasizes the importance of this aspect by 
stating that the basic trade-offs in underwater imaging design are between camera- 
light separation, contrast, and power. In fact, the rough classification of underwater 
imaging systems provided in Jaffe (1990) is widely referenced by the underwater 
vision community. According to this classification, underwater imaging systems 
fall into one of the following three groups: 

1) Conventional systems: The light source is placed close to the camera. This 
configuration makes it possible to obtain images up to two attenuation lengths. 

2) Increased camera-light separation: The light source is separated from the 
camera as much as possible in small underwater vehicles (3-5 m). This 
configuration allows the acquisition of images up to three attenuation lengths. 

3) Exotic systems: These systems can extend the visibility by using range gated 
or laser based systems, but this increases their complexity and price, and also 
their power consumption. 

Thus, increasing the separation between the light source and the camera, 
whenever the system frame allows it, is a very simple and cheap way to obtain 
images one attenuation length farther than conventional systems, neither involving 
major changes in the imaging architecture nor requiring more power. Although 
this idea has been widely used and is considered common knowledge, very few 
studies have quantified the effects of such an increase in separation. Jaffe models 
the behavior of light in water as well as the image formation process and performs 
several computer simulations in Jaffe (1988, 1990). These simulations assume a 
camera pointing to the sea floor and evaluate the image contrast for different 
separations between the camera and the light source, moving the light source 
vertically and horizontally. 

Simulations show that separating the light source horizontally dramatically 
increases the image contrast. Separations from 10 to 20 m considerably increase 




Figure 2.2 (a) Conventional system and (b) increased camera to light source separation. 

The darker area depicts the volume of water where backscatter appears. 
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the contrast, but it tends to slightly decrease at large distances, mainly due to 
the light attenuation. For example, for a camera altitude of 40 m, increasing the 
separation from 3 to 5 m results in an approximate doubling of the image contrast. 
The results provided by Jaffe (1990) are summarized in Figure 2.3. The image 
contrast also increases when separating the light source vertically. This may seem 
counterintuitive, as this kind of separation does not decrease the volume of water 
where backscatter may appear. According to Jaffe, this improvement appears 
because the area directly in front of the camera is not illuminated as intensely as in 
the case where no separation exists. 

Nevertheless, the improvements in image contrast are much smaller when 
moving the light source vertically than horizontally. Increasing the horizontal 
separation between the light source and the camera is a worthwhile improvement 
for some underwater imaging systems, but it can be difficult to apply in vehicles, 
depending on their shape and size. However, it is always possible to place the light 
source and the camera in different vehicles and coordinate their navigation (Jaffe, 
2007). These systems increase cost and complexity in their coordination and can 
generate moving shadows in the image. 

2.3.3 

Illuminating Systems 

When choosing a suitable light source for an undersea imaging installation, a 
balance has to be struck between the cost and the quality of the light emitted. 
These terms are all reflected in the parameter efficacy (lumens per watt). Efficacy 
is an important term, especially in autonomous vehicles that need to optimize the 
power consumption to generate the maximum light. Underwater vision systems 
have progressively evolved toward the use of more effective technology, and that 




Figure 2.3 Contrast transmittance versus separation: (| | |) Vertical separation. (— — — ) 
horizontal separation. Source: © [1990] IEEE. Reprinted, with permission, from jaffe (1990). 
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obviously includes the illuminating systems. Different types of light emitters with 

diverse levels of efficacy are listed below: 

• Halogens: These use a filament to ignite halogen gas. Halogen lamps are more 
effective than incandescent ones since they emit a whiter and, on average, a 30% 
more brilliant light consuming less watts and irradiating less infrared (IR) heat. 
Halogens do not produce blackening of the bulb glass in usage and they are 
smaller than a standard incandescence emitter. Halogens have a longer lifetime 
than standard incandescence lamps. 

• HID (high intensity discharge) lamps (also known as xenon lamps): These lamps 
use an electrical discharge between two electrodes to ignite xenon gas in a sealed 
bulb. They emit a blue-white light closer to natural daylight. This technology 
is ten-times better than halogens in terms of durability and security, and also 
increases efficiency, producing much more light (measured in lumens or lux) 
than halogens with the same power consumption and emitting much less heat. 
HIDs also emit light that penetrates darkness better than halogens. HlDs are 
extensively used in underwater applications such as ROVs (where the power can 
be supplied via its umbilical) or static underwater stations. 

• HMI (hydrargyrum medium-arc iodide lamp): In these lamps, bulbs are filled 
with mercury vapor that is excited by creating an electrical arc between two 
electrodes. They are useful in imaging systems that need a long-time light 
exposure. They are typically used in the film industry, but since 1996 they have 
been used in underwater photography. They are usually expensive and are high 
power consumers (>600 W) although some manufacturers are beginning to 
produce low consumption HMI lights. They produce extraordinary illumination 
quality and are used in some ROVs for underwater filming at great depths. 

• HIF (high intensity fluorescent): In many cases, these systems are more energy- 
efficient than HID solutions. They generate lower lumen depreciation rates, 
present better dimming options, and reduce the glare. They are able to illuminate 
a more extensive area than HID lamps do. Fluorescent solutions are applied 
underwater to attract fish but rarely for underwater vehicles. 

• LEDs (light emitting diodes): These systems are supplied with direct and contin- 
uous current (12 or 24 V). The power consumption is four-times less, on average, 
than with halogens. LEDs can emit light in a wide range of wavelengths, they 
have a small spectral bandwidth, their lifetime is much longer than halogens, 
and, in general, they have a higher efficiency than HIDs. Although they can be 
expensive, LEDs have become especially important and suitable for underwater 
autonomous vehicles equipped with batteries since they considerably reduce the 
power consumption. 

• Infrared: Light is considered to be infrared for wavelengths in the 750 nm 
to 1 mm range. Infrared imaging is extensively used for night vision and in 
some underwater applications. Infrared light emission must be accompanied by 
the use of cameras with infrared receivers (IR cameras). Infrared light is very 
useful for filming marine fauna, since it can be used to remotely determine 
the temperature of targets and it is not visible. Infrared light penetrates less 
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than orange, yellow, or green light, but it has been demonstrated to be suitable 
for some underwater imaging applications (Lam et al, 2007; Sedlazeck, Koser, 
and Koch, 2009). In some cases, the undesired scatter can be filtered-out in 
the receptor if the wavelength of the received energy is different from that the 
emitted one. 

• Laser: Lasers usually operate in the 350-630 nm range. They are usually employed 
in underwater extended range techniques, since they permit illumination at 
greater distances than light with lower or higher wavelengths. Lasers, though, 
also allow the calculation of distances and object sizes with a high degree of 
precision. Lasers have two main problems: (i) emitters are considerably more 
expensive than HID, LEDs, or infrared and (ii) emitters generate a very narrow 
beam, which makes it necessary to scan the environment to take an image, 
thereby delaying the process and reducing the frame rate. 

Although these systems are considerably more expensive than conventional 
incandescent lamps, they should be considered due to the wide range of benefits 
that they offer. 



2.4 

Laser-Based Techniques 

Techniques using laser emitters and receptors are intended to considerably extend 
the range of the captured scenes thanks to the laser scattering and absorption 
properties, improving contrast and resolution beyond that offered by other means 
such as infrared or systems with visible light (Funk, Bryant, and Beckman, 1972). 
Laser emitters concentrate intense light over a very small and narrow area, but this 
light propagates longer. However laser-based solutions are difficult to implement, 
they need important post-processing software, and can be more expensive than 
standard solutions. They are the best choice for long range imaging since they 
can be effective up to five, six, or seven attenuation lengths, while with visible 
light images are clear up to two-three attenuation lengths. The most outstanding 
techniques involving laser infrastructures are detailed below. 

2.4.1 

Laser Range-Gating (LRC) Methods 

These systems scan the scene with laser pulses emitted at frequencies lower than 
100 Hz (short pulses of approximately 6ns). Special cameras must be provided 
with Intensified CCD (ICCD) sensors that can synchronously capture laser light in 
time gates. Knowing the speed of light in water and the distance to the target, the 
time that the light pulses need to travel from the emitter to the target and return 
to the receptor can be easily calculated. Therefore, using a gateable receiver with a 
sufficiently high temporal resolution, most of the undesired scatter can be filtered 
out leaving only the signal returned by the target. In 1994, range-gating was already 
a consolidated technique for extending the underwater imaging range (Swartz, 
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1994; Weidemann et al., 2005). More recently, Han et al. (2009) proposed a new 
method for synchronization control of the laser emitter and the camera receptor 
in a LRG system. Improving the synchronization between the emitter and receiver 
helped to greatly improve the image resolution (He, Li, and Zhou, 2009). 

Recently, Wu et al. (2009) have simulated and evaluated the performance of gated 
ICCD cameras to be applied in LRG-based underwater imaging applications. 

2.4.2 

Laser Line Scan (LLS) Methods 

Laser line scan (LLS) systems scan the environment with a narrow laser beam 
perpendicularly to the direction of the sensor support platform and sweeping out 
light rays as the vehicle moves. LLS techniques can use either continuous or pulsed 
laser reducing the backscatter effect by displacing the receiver from the laser source. 
Pulsed LLS can reduce the undesired scatter by time gating the receiver aperture 
to capture the light only at certain time intervals. Caimi et al. (2007) and Dalgleish 
et al. (2008) demonstrated that pulse-gated LLS provide a much better performance 
than previous continuous wave {CW) LLS configurations. CW-LLS systems, under 
certain conditions, are limited by multiple backscattering caused by an increase of 
turbidity or an increase of the illumination distance (Dalgleish et al., 2009). 

LLS systems have recently been considered as the optimal technology for extended 
range underwater imaging. 

2.4.3 

Scattered Light Rejection Using Modulation/Demodulation Techniques 

Modulation/demodulation consists of displacing the frequency spectrum of a 
signal, and it can be used, for example, to discriminate useful components from 
noise, or to transmit several digital channels in a unique physical infrastructure. 
Coherent demodulation uses a receiver equipped with a phase-locked-loop (PLL) 
that tracks the phase and the frequency of the received signal. The fact that 
seawater particles cause an important dispersion of the light (especially at high 
optical frequencies) has motivated many researchers to question if coherent/non- 
coherent modulation/demodulation techniques could be efficient in underwater 
imaging systems. Illumination with laser or infrared light (amplitude modulated 
at lOOMHz, typically) has the advantage that the frequency, wavelength, phase, 
or amplitude of the emitted light are known and can be demodulated in the 
receptor separating the signal spectrum from the scatter frequency components. 
Infrared light has been scarcely considered because it is absorbed by water at 
shorter distances than laser light. Other systems demonstrated that it is possible to 
enhance contrast and image quality at large distances by coherently modulating and 
demodulating the illuminating laser signal in phase (B-PSK or Q-PSK; Cochenour, 
Mullen, and Laux, 2007; Laux, Mullen, and Cochenour, 2007). 
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2.5 

Underwater Imaging Infrastructures 

The following review of previous research and practical applications will assist 
with the design of optimal systems for underwater imaging. Many of the existing 
systems seek to improve images by using several cameras for different purposes 
and/or various light sources: 

1) Eye-in-the-sea: This is an unobtrusive deep-sea observatory and uses a LED 
that emits red light (wavelength 680 nm) in combination with a low-light-level 
(LLL) camera (Widder, 2007). 

2) Rosenkranz et al. (2008) present an underwater imaging system designed for 
fisheries detection and observation. A gigabit Ethernet high-resolution camera 
is connected via optical fiber to the control computer. The environment is 
illuminated with strobe light to eliminate motion. 

3) Lam et al. (2007): These authors have built an underwater camera system 
for monitoring marine fauna in coral reefs. The system is provided with 
a wide-angle JR-sensitive camera connected via optical fiber to an on-land 
controlling computer. 

4) The submarine observatory NEPTUNE (Pacific Ocean) needs to integrate 
station nodes and static imaging systems to support ROVs in their surveillance 
tasks. Imaging platforms have been equipped with a high-definition camera, 
and three different light sources: (i) a three beam laser system to provide range 
information to the user, (ii) two dimmable LEDS with more than 406 Im with 
an equivalent power of 250 W, and a life of 50000 h, and (hi) a 150 W HID 
light to extend the range of visualization (Roston, Bradley, and Cooperstock, 
2007). 

5) Sedlazeck, Koser, and Koch (2009) have presented a 3D reconstruction 
system based on feature tracking. Images were provided by the ROV Kiel 
6000, equipped with three cameras: (i) a high-definition camera, (ii) a standard 
color PAL still camera and, (hi) a slave-mode controlled camera with automatic 
flash shot. The ROV is also equipped with a 250 W halogen lamp, another 
70 W HID light and a 400 W HMI dimmable focus. 

6) Negahdaripour and Firoozfam (2006) have presented a vision system for 
automatic ship hull inspection, applicable to AUV but firstly tested with 
images provided by a ROV. The ROV was equipped with a stereo camera. 

7) Negahdaripour, Sekkati, and Pirsiavash (2007) have explored the possibility 
and results of using optical-acoustic stereo imaging for 3D shape recovery 
of underwater targets. DIDSON (2009) acoustic cameras provide acoustic 
images with such a high degree of reliability that they can be analyzed and 
processed in the same way as standard images. 

8) The CSIRO Marine and Atmospheric Research Division in Australia (Shortis 
et al., 2007) has developed a towed platform for sea bottom map construction, 
fisheries study, detection, and surveillance. The system was equipped with 
two standard PAL cameras in stereo configuration for video recording that 
could be accurately geo-located thanks to the vessel Differential GPS (DGPS) 
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equipment and to the navigation data and to ultra-short baseline (USBL) 
tracking beacon data on the towed body. Additionally, a high-resolution still 
camera for computing mosaicking and a forward-looking camera for obstacle 
detection and avoidance were also available. For illumination, two 250 W 
incandescent lamps were used for the video recording and two strobes for the 
still camera. 

9) Some years ago, an underwater docking station was developed to enhance the 
performance of the AUV REMUS (Allen et al, 2006). The vehicle incorporated 
several cameras for different purposes: (i) one Ethernet video camera to 
provide real-time information about the process of entering and leaving the 
docking station and (ii) a periscope camera for sea surface observation. This 
camera was mounted in a housing that could be deployed and retracted above 
and below the sea surface. 

10) Hercules (IFE, 2009) is a ROV especially designed for working in the deep 
sea, manipulating, recovering, or digging in ancient shipwrecks. Hercules 
is equipped with one high-definition video camera for monitoring the sea 
bottom and two still cameras for mosaicking tasks. 

11) ARGUS is a ROV produced by Woods Hole Marine Systems, Inc. (Woods 
Hole Oceanographic Institution) that can operate independently or as a 
partner of other ROVs and it is equipped with cameras and HMI lighting for 
underwater image registration (WHMS, 2009). 

12) Phantom DHD2+2 and Phantom HD2+2 [DOE (Deep Ocean Engineering) 
(uww.deepocean.com)] are two ROVS designed for fisheries and scientific 
research, military missions, gas/oil pipes inspection, or underwater filming, 
and are especially suitable for deep water and strong currents. Both vehicles 
incorporate a high-resolution PAL/NTSC color camera with a wide-angle lens, 
and two 250 W halogen lamps to illuminate the area of inspection. 

13) SubSea7/Seebyte AIV (Autonomous Inspection Vehicle)(umw.subsea7.com): 
Although this is still a prototype and it has not been yet commercialized, the 
cooperation of both companies has lead to the presentation of a new AUV for 
inspection and intervention on offshore oil and gas installations. This vehicle 
will carry sonars and cameras powered by the robot batteries. 

2.6 

Image Improvement via Polarization 

As explained above, a fraction of the light that passes through water is backscattered 
to the camera before reaching the target, significantly reducing the contrast of the 
resulting image. The effects of backscatter depend on the amount of water in the 
line of sight and, thus, on the distance to the object being observed. Moreover, 
the backscatter is magnified when artificial illumination is used (Treibitz and 
Schechner, 2009a). As artificial illumination is a common requirement for sub-sea 
operation, backscatter is an important problem in underwater robotics. 
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Different authors have pointed out that the media responsible for the backscatter 
and the objects in the environment behave in different ways when they are exposed 
to polarized light. However, there is no consensus in the research community 
regarding this subject. According to Treibitz and Schechner (2006) some studies 
assume that objects in the scene preserve polarization while backscatter does not, 
and some other studies state that polarization is preserved only by the backscatter. 
Despite these opposed points of view, polarization can be used to reduce the 
negative effects of backscatter because, either in one sense or another, media and 
scene objects have different responses to polarized light. 

This section focuses on describing the potential benefits of using polarization 
to increase contrast in underwater images and shows some tests conducted to 
determine its usefulness for applications in underwater robotics. It is beyond the 
scope of this chapter to explain the basic concepts of polarization, so the reader is 
referred to the papers authored by Schechner that are cited in the references section. 

Although uncommon, light with a high degree of polarization (DOP) can be 
found in nature. For example, according to Waterman (1954), sunlight becomes 
polarized in underwater environments when traversing the interface between air 
and water. Besides, unpolarized light can be polarized by means of a polarizing 
filter. The cost of this transformation is a very important loss of light energy. 
Off-the-shelf polarizing filters reduce the light intensity by more than 50%. 

2 . 6.1 

Extended Range Using Polarization 

The different interactions of objects and backscatter with polarized light can be 
used to improve contrast in underwater imaging. Some studies exploit this idea in 
underwater environments where sunlight is sufficient and no artificial illumination 
is required (Karpel and Schechner, 2004; Schechner and Karpel, 2004, 2005). These 
studies are based on the assumptions that, first, the interface between air and water 
partially polarizes the sunlight and, second, that backscatter preserves polarization 
while the observed objects do not. The authors’ proposal is to attach a polarizing 
filter to the camera and capture two images of the same scene using orthogonal 
polarization angles. By properly combining the resulting two images, backscatter 
can be significantly removed. This idea is illustrated in Figure 2.4a. Similar 
approaches have been proposed (Morgan, Khong, and Somekh, 1997; Treibitz and 
Schechner, 2009a) to deal with those underwater environments where sunlight is 
not sufficient. In these cases, the artificial light source is fitted with a filter so that 
the scene is illuminated with polarized light (Figure 2.4b). 

2 . 6.2 

Housing 

Another important issue when designing an underwater imaging system involving 
polarization is related to the interface of the camera with the water, as it should 
have a minimum effect on the polarization (Karpel and Schechner, 2004). First, 
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Figure 2.4 Extended range using polarization: (a) with sunlight and (b) with artificial light. 



the housing must guarantee that the only light going through the camera lens 
comes from the viewing port. Other light sources must be blocked. Second, the 
stress in the transparent port’s material changes the polarization of the light that it 
transmits due to the so-called photoelastic effect. To reduce this effect, glass ports 
are recommended. Regarding its geometry, even though flat ports are acceptable, 
the optimal choice is a dome whose center coincides with the center of projection 
of the camera lens. 

2.6.3 

Experimental Evaluation 

To evaluate the advantages and drawbacks of using a polarization-based system 
underwater, we have implemented the proposal in Treibitz and Schechner (2009a) 
and tested it in a water tank in our laboratory. As stated previously, this proposal 
consists of attaching a polarizing filter to the artificial light source and another 
polarizing filter to the camera (Figure 2.4b). Then, one image is taken using 
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orthogonal states for both polarizers and a second image is obtained with polarizers 
parallel. By combining both images, the effects of backscatter should be reduced. 

The water tank used was 1.5 m x 0.4 m x 0.45 m and contained 2401 of water. 
The only light source during the experiments was a 3 W LED lamp fitted with a 
linear polarizer. The images were taken using a 10 megapixel reflex camera. Shutter 
speed, aperture, focus, and any other camera parameter were manually adjusted 
to optimize the quality of the images gathered. Calibrated sheets, as well as some 
other objects, ranging from 0.5 to 1.1 m to the camera lens, were placed inside the 
water. To cause water turbidity, various amounts of milk were dropped into the 
tank. Figure 2.5 illustrates the experimental setup. 

Some of the results are given in Figure 2.6, where two images taken for each 
scene, as well as the image resulting from the scatter reduction process, are shown. 
The figure shows the results for two different levels of water turbidity. As can be 
seen, the reconstructed image contains less backscatter than the corresponding 
originals. However, a detailed analysis shows that the resulting images have more 
noise and are saturated in some areas. This observation is consistent with Treibitz 
and Schechner 2009a and 2009b, who state that keeping the camera’s exposure 
time constant, which is usual in video acquisition, never improves the signal to 
noise ratio (SNR) of the resulting images. Moreover, they also showed that even 
the use of automatic exposure and assuming optimal exposure times only improve 
SNR in rare cases. 

Clearly, the question to answer here is whether polarization-based methods 
provide sufficient benefits, compared to simply using a camera with no filters and 
an unpolarized light source. Comparison with other more complex and expensive 
methods, such as laser based ones, was not the aim of these experiments. 

The main problem found is that the double filtering process needed by this 
method strongly attenuates the light. In most cases, low illumination of the scenes 
increases the SNR in digital cameras. To minimize this effect, the lighting power 
should be increased, reducing the energy available for the complete system. This 
fact strongly restricts the usefulness of this method in limited energy platforms 
like AUVs. In addition, two images per scene, obtained from the same camera 




Figure 2.5 Experimental setup. 
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(e) (f) 

Figure 2.6 Some results using polarization: (a) and (d) light and camera polarizers per- 
pendicular; (b) and (e) light and camera polarizers parallel; (c) and (f) reconstructed 
images; (a)-(c) with 20cc of milk and (d)-(f) with 50cc of milk. 

position, are needed. Thus, if the camera is attached to a moving platform, it is not 
possible to guarantee this premise. Moreover, using two images per scene reduces 
the available frame rate by half, which is a strong restriction if vehicle or object 
motions are to be tracked. Other approaches, such as polarized light stripping 
(Gupta, Narasimhan, and Schechner, 2008) and polarization vision (Wolff, 1997) 
are also useful in removing backscatter at the cost of additional, expensive hardware 
in the first case and the requirement of three images per scene in the second case. 



2.7 

A Vision System for Underwater Applications 

Having analyzed the fundamentals of underwater image formation and some of 
the most accepted imaging techniques, a new vision system being developed at the 
University of the Balearic Islands is now presented. First, the hardware structure 
and the software architecture are introduced and, then, different configurations 
of the system and some recent experiments carried out with it are described. The 
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vision system has been conceived to provide visual perception to an AUV with 
intervention capabilities (I-AUV). The main specifications and limitations that have 
conditioned the resulting design are listed below: 

• mechanical: maximum volume 91, neutral buoyancy, minimum depth rating 
100 m; 

• electrical: voltage source 14.4 V and SOW of maximum power consumption 

• perception needs: image resolution 640 x 480 or higher, minimum frame rate 
ten frames per second, possibility of 3D reconstruction; color images optional; 
two cameras could be connected if necessary; 

• communication: wired Ethernet link to other modules of the vehicle of 100 Mbps 
or higher. 

Another requirement that was taken into account was the configuration flexibility 
of the system, so it would permit different geometries and easy adaptation to 
different underwater platforms. In addition, during development of the system, 
some changes were introduced, mainly concerning mechanical aspects, to enable 
stand-alone operation, thus, creating a micro-vehicle that could be operated in 
shallow waters at a reasonable cost. 

2 . 7.1 

The Fugu Vision System 

The Fugu vision solution is based on two stereo rigs and a computer system linked 
through a firewire bus. When the system is configured in iis flexible option, which 
is called Fugu-F, each module is placed in an independent watertight case rated 
for up to 100 m depth. Each camera is connected to the computer cylinder through 
an independent waterproof cable and a connector’s set. Other connectors, visible 
on the cylinder cover (Figure 2.7a) are used for power supply and communication 
purposes. Fugu-F is a modularized solution that allows different cameras to be 
used and a variable geometry setting, depending on the mission to be carried out 
and the power and payload available. Both configurations shown in Figure 2.7 
are using Bumblebee2 cameras, but any other firewire compliant devices could be 
easily adapted. 

Identical hardware used for the Fugu-F version has been employed for a more 
compact solution, called Fugu-C, by substituting the aluminum cylinder for a trans- 
parent container and adding horizontal and vertical propellers (Figure 2.7b). Fugu-C 
can carry its own batteries and, at present, it can be remotely operated through a 
thin data cable. An autonomous navigation module that, thanks to a MEMS (Micro- 
ElectroMechanical System) based inertial unit and to visual information, will allow 
the vehicle to follow short programmed paths is under development. This new 
setting provides a solution for underwater image recording and observation, up to 
depths of 15 m, at moderate cost. 

The software control architecture of the Fugu vision system is based on the 
robot operating system (ROS) middleware (Quigley et al., 2009). Thus, services 
or capabilities are managed as messages and modules compartmentalized and 
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(b) 

Figure 2.7 (a) Fugu-F system, the stereo cameras in its watertight case and the cylinder 

containing the computer system; (b) Fugu-C, a compact and auto propelled configuration 
with identical vision functionality. 

implemented as nodes. All the vision related services are run on the Fugu computer. 
Other capabilities like navigation, mapping, or guidance, among others, can reside 
on the same computer or on other independent hardware, depending on whether 
the compact or the flexible setting is implemented, respectively. Most of the basic 
vision services like visual feature detection and tracking, motion estimation, target 
identification or image tagging, and storage, have already been implemented. 
Other functionalities, especially regarding 3D information processing, are still 
under development. 

Fugu has already been tested on different vehicles. First, the Girona-500 (Ribas 
et al., 2012) carried the Fugu-F with one camera during a set of autonomous 
search and recovery experiments made in sea conditions (Garcia et al., 2012). The 
Girona-500 is an AUV developed at the University of Girona as part of a Spanish 
research project aimed to design and test new methodologies for multipurpose 
autonomous intervention tasks (Prats et al., 2012). During the above-mentioned 
trials, some important functionalities of the vision system were tested. First, when 
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the vehicle was traveling close to the sea floor, the visual odometer function 
provided a high rate of motion estimate to the navigation system thanks to the 
homography obtained from frame to frame. During the search step, the target 
detection module was in charge of determining if the object to be recovered was 
observed in the field of view. Finally, the target tracking function fed the station 
keeping and the arm controller to guide the vehicle while maneuvering during the 
recovery stage. Figure 2.8a shows the Girona-500, equipped with the arm and the 
Fugu-F. The white box placed at the bow of the lower hull is the downwards-looking 
camera. Next to the camera, a cylindrical case can be observed, which corresponds 
to the port LEDs based focus. Another identical focus is placed on the starboard 
side. The computer cylinder was placed in one of the upper hulls of the vehicle. 

Some more experiments have been made in coastal waters using both login 
visual and acoustic images to analyze their correspondence and complementary 
information. In these experiments Nessie-VI was the vehicle used, equipped with 
the Fugu-F system and a multibeam imaging sonar (Figure 2.8b). Nessie-VI is a 
torpedo shaped AUV developed by the Ocean Systems Laboratory at Heriot-Watt 
University. During these experiments, the Fugu-F cylinder could not be installed 
in the hull, so it was placed just underneath it. The stereo camera was placed at the 
bow area in a downward looking and slightly forward orientation to maximize the 
overlapping area with the imaging sonar. 

The main objective of the experiments with Girona-500 and Nessie-VI was to 
test for the first time the Fugu-F system under real conditions, that is, integrated 
as part of the equipment of an AUV and in the sea. Consequently, attention was 
focused on verifying technical and operative aspects like power supply stability, 
non-interference with other on board equipment, response of the vision system 
functions, communication issues, and proper integration of the whole system from 
the mechanical and electrical point of view. In fact, a complementary result of the 
experiments is the recording of a few hours of video gathered synchronously with 
the navigation data of the vehicles. This information is under study to determine the 
best configuration of the vision equipment and to develop a vision-based navigation 
system to improve the autonomous capabilities of the AUV when it operates at a 
short distance from the sea floor or to the target. 

Finally, the Fugu-F can also be equipped with its own batteries and, thanks to 
the hard drive housed in the cylinder, operate in a stand-alone mode. Thus, the 
system can be installed on a submarine fixed position and remain deployed from 
several hours to a few days depending on the functions being executed. 



2.8 

Conclusion 

Owing to the interaction between water and light, sub-sea images have special 
characteristics that should be taken into account when designing an underwater 
optical imaging system. Aspects such as the illumination type or the camera’s 
housing shape and material, which are generally not relevant in many terrestrial 
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(b) 

Figure 2.8 Two different vehicles carrying Fugu-F on board: (a) the Girona-500 and (b) 
Nessie-VI. Photograph: David March Morla, reproduced with kind permission. 
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applications, become important when going under water. If the imaging system 
has to be mounted on a ROV or on an AUV, some constraints like limited 
power supply, weight or size become critical. In addition, human intervention is 
often undesirable or impractical in many underwater scenarios. In these cases 
a fully autonomous operation is required. This work presents most aspects to 
consider when designing an imaging system for underwater applications. First, the 
macroscopic water-light interaction phenomena have been described, and then 
the illumination techniques available and most of the known imaging systems 
were reviewed. Next, a polarization-based technique to improve images in turbid 
waters was analyzed and some experiments carried out at the authors’ laboratories 
described. As a consequence of all the aspects previously mentioned, a new vision 
system, called Fugu, was presented. The system has been designed to satisfy 
the operational requirements of a new AUV with intervention capabilities under 
development in the context of the Trident project. In addition to the specifications 
imposed by the project, Fugu has been designed to be highly adaptable to different 
applications. Thus, two versions of the system have been described and some 
recent trials reported. 
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Holographic Microscopy of Marine Organisms 

Stefan K. Jericho, Manfred H. Jericho, and Hans J. Kreuzer 



3.1 

Introduction 

Zoo- and phytoplankton consist of populations of organisms ranging in size 
from copepods (up to several millimeters) to bacteria (which may be sub-micron). 
Plankton forms the foundation of all marine communities, from the continental 
shelves to coral reefs. The health of the complex ecosystems in these habitats is 
fundamentally dependent on a well balanced population of plankton. To obtain 
an understanding of population growth, prey-predator interactions, and nutrient 
transfer along the food chain requires observations and measurements on plankton 
populations over extended time periods and at many levels of the water column. 
To capture organism movement and feeding characteristics requires the direct 
imaging of organisms in their marine environment. This is particularly challenging 
for imaging of nano- and picoplankton, which includes cyanobacteria and small 
algae, since their size can be close to the theoretical resolution limit of optical 
imaging techniques. Microorganisms in this size range may play an important role 
in decomposition of organic matter and in the recycling of nutrients (Ferrier-Pages 
and Gattuso, 1998). Coral reefs, for example, have very productive ecosystems 
but exist in carbon-deficient sea-water. It has been suggested that dissolved and 
particulate organic matter is recycled by bacteria and thus transformed into bacterial 
biomass and then transferred to higher trophic levels by flagellates and dilates 
(Sorokin, 1973). Imaging of organisms in this size range can greatly assist studies 
on the mechanisms at work at the bottom of the food chain. To address the 
need for high-resolution imaging in a marine environment, several groups have 
developed submersible holographic microscopes. This chapter reviews some of 
these instruments that have been successfully employed to obtain in situ images of 
plankton organisms ranging in size from micro- to picoplankton. 



Imaging Marine Life: Macrophotography and Microscopy Approaches for Marine Biology, First Edition. 

Edited by Emmanuel G. Reynaud. 

© 2014 Wiley-VCH Verlag GmbH & Co. KGaA. Published 2014 by Wiley-VCH Verlag GmbH & Co. KGaA. 



50 I 3 Holographic Microscopy of Marine Organisms 

3.2 

Advantages of Holographic Microscopy 

Holographic microscopy has several advantages over conventional optical 
microscopy. In conventional microscopy, high resolution can only be obtained 
with high numerical aperture objectives. This implies a very small depth of field 
(1 pm and less) and it is therefore impossible to image organisms in motion such 
as marine plankton. The remarkable thing about holographic imaging is that a 
single hologram contains information about all objects in the illuminated volume 
that scatter light. Images of the objects are then obtained by applying direct optical 
(film) or digital (CCD, charge coupled device) reconstruction methods in selected 
planes in the imaged volume (Xu et al., 2001). With a sufficient number of planes, 
a complete 3D view of the object distribution can thus be obtained. When a 
succession of holograms is recorded at video rates, trajectories and the whole 
dynamics of the object distribution can also be captured (Xu et al., 2003). 

Object illumination can be achieved either with plane wave fronts from collimated 
light or with light from a point source giving spherical wave fronts. In the latter 
case the hologram encodes magnified images of objects. Since holography is based 
on interference, the more interference fringes one can capture with the recording 
medium the higher the obtainable resolution. Holographic films can capture 
fringes spaced less than a micron apart and are a recording medium with the 
highest potential resolution. However, digital recording technology has advanced 
enormously in recent years and the many advantages of CCD chips have greatly 
made up for their smaller dimensions and larger pixel size. 

In Section 3.3 we give a brief review of recent efforts by several groups to 
image marine plankton from centimeter size copepods to <50 pm diameter 
microplankton using collimated beam holography. The highest resolution can 
be obtained with point source digital in-line holographic microscopy (DIHM), 
where objects are illuminated with nearly spherical waves from pin holes with 
diameters of 0.5 pm and smaller. This allows imaging of nano- and picoplankton 
down to the size of bacteria, and will be discussed in some detail in Section 3.4. 
Plankton studies with holographic microscopy can be divided into three main 
parts: (i) recording of high-quality holograms; (ii) reconstruction of holograms 
either optically or computationally to retrieve from the holograms in focus images of 
objects distributed throughout the sampled volume; (iii) evaluation of reconstructed 
sample volumes. This last point includes identification of objects, determination 
of the number density of objects in various size categories, construction, and 
evaluation of particle trajectories, and the correlation of all these with other 
parameters of interest such as depth, temperature, and salinity of the water. The 
large amount of information contained in a single hologram thus generally means 
that automated methods for image evaluation have to be employed. In the following 
we will only describe instrumentation and methods that have been used to (i) record 
and (ii) reconstruct holograms of in situ imaging of marine plankton. 
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3.3 

Past Attempts to Image Microplankton 

An early attempt at holographic imaging of marine organisms was made by 
Knox and Brooks (1969) with an in line cine-holographic recording system. This 
system was converted by Heflinger, Stewart, and Booth (1978) for off-axis recording 
of holograms (Figure 3.1a. A collimated beam of coherent radiation illuminated 
plankton in an aquarium tank and a relay lens and off-axis reference beam produced 
a hologram on 35 mm movie camera film capable of 200 frames s"^. Recordings 
were carried out with a pulsed argon-ion laser (514 nm) or a xenon laser (535 nm). 



Collimated beam 




35 mm movie camera 



Reference wave 




Figure 3.1 (a) Holographic off-axis imaging 

system originally used by Heflinger, Stewart, 
and Stewart (1978). A collimated laser beam 
illuminates objects and produces, with the 
help of a collimated reference beam, a holo- 
gram on a 35 mm film, (b) Collimated beam 
in-line holography used by Malkiel, Alquad- 
doomi, and Katz (1999). Holograms from 
a large sample volume are projected on a 
cine film by relay lenses, (c) Point source 
in-line holographic microscopy as used by 



jericho et al. (2006). A microscope objective 
illuminates a pinhole situated in a pressure 
chamber. The nearly spherical waves that 
emerge from an exit window illuminate the 
sample. The reference wave, together with 
scattered radiation, produces a magnified 
hologram on a CCD or CMOS chip situated 
in a second pressure chamber. The cham- 
bers are rigidly coupled together but allow 
fluid to pass freely between them. 
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Hologram reconstruction was achieved by passing a conjugate reference beam 
back through the lens and a column of water in a water-filled variable thickness 
aquarium compensation cell. This way all aberrations introduced during recording 
are canceled. Their system achieved sufficient resolution so that 2-3 pm diameter 
spines on a marine diatom could be recognized. 

In situ imaging of live plankton became possible when submersible holographic 
recording systems were deployed. The early systems developed by groups lead by 
Malkiel, Alquaddoomi, and Katz (1999), Craig et al. (2000), and Watson et al. (2001) 
all recorded holograms on high-resolution film. 

The system of Malkiel, Alquaddoomi, and Katz (1999) used collimated beam 
in-line holography. The principle of this is shown in Figure 3.1b. A collimated 
ruby laser beam illuminated a volume of sea water 287 mm long and 63 mm in 
diameter. Mirrors and a relay lens then allowed the recording of the hologram on 
high-resolution 70 mm holographic film at a maximum rate of one hologram every 
30 s. The HoloCamera system was mounted on a manned submersible, the Johnson 
Sea Link (Liberatore et al., 1997). This HoloCamera could record 400 holograms on 
slow buoyancy driven ascents or when moving at 1 kt during powered transects. 

Collimated beam in-line holography and high-resolution film recording allows 
imaging of liters of sampled volume. The retrieval of information that is stored 
in the hologram can, however, be time consuming and requires two basic steps. 
The film has to be developed and the hologram must be reconstructed. Hologram 
reconstruction for the Malkiel, Abras, and Katz (2004) holocamera was accom- 
plished with conjugate collimated He-Ne illumination of the holograms. The same 
relay lenses that were used in the hologram recording were also used to bring the 
real image volume close to a digital camera (3.59 pm per pixel) equipped with a 
Micro-Video-Zoom-Objective and a 0.75 x sub-lens. The camera was mounted on 
a three-axis motorized traverse. The sample volume was scanned with 2 x magnifi- 
cation at 14 X 14 camera positions in each recording plane. Plankton information 
throughout the imaged volume was obtained in this way for 200 planes with succes- 
sive planes separated by 1 mm. The 1 mm spacing between reconstruction planes 
is much larger than the dimensions of most plankton organisms in a sampled 
volume. Nevertheless, this coarse slicing of a single scanned reconstructed volume 
resulted in about 35 GB of image information. Since hologram recording and image 
reconstruction were performed with different wavelengths, the reconstructed vol- 
ume was compressed by 18%. Despite these complications, image resolution was 
adequate to detect particles down to 10 pm in diameter. For particles smaller than 
50 pm in diameter, improved resolution was obtained through band-pass filtering 
by insertion of a pin and iris filter at the focal point of the first relay lens. With 
the holocamera of Katz’s research group, the first in situ high-resolution images 
of marine organisms such as calanoid copepods and ctenophores were obtained 
so that information about plankton distribution, such as particle density and its 
variation with depth, was obtained for the first time with holographic imaging 
(Malkiel et al., 2006). 

As part of a multinational project funded by the European Commission under the 
Mast 111 initiative, a holographic recording system, the HoloCam, was developed 
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by the group lead by Watson et al. (2001). The objective of the project was to develop 
a holographic imaging system to analyze and monitor the populations of living 
organisms and inanimate particles within the world’s oceans. The laser in their 
HoloCam instrument was a frequency doubled Nd:YAG laser (532 nm) capable of 
700 mj single pulse power and a pulse duration of 10 ns. The laser beam was split, 
with one beam (100 mJ) performing collimated in-line holography (Figure 3.1b) 
while the other beam (600 mJ) was used to image the same water column in 
an orthogonal direction using an off axis reference beam. The water sample for 
off-line holography was illuminated through three specially designed light rods. 

All holograms were recorded on Agfa Millimask high-resolution silver halide 
photographic emulsions supported on glass plates (102 mm^). Holograms could 
be collected at 10 s intervals. The combination of in-line and off-line holography on 
one instrument allowed recording of very large volumes with 3.51 for the in-line 
part and 501 for the off-line system. During three dives to a depth of 100 m at 
Bonawe Deep in Loch Etive (a sea loch in the west of Scotland) 107 holograms were 
recorded. 

Hologram reconstruction was performed with conjugates of the original refer- 
ence beams, with beams normal to the photographic plates for in-line holograms 
and at 60° for off-line holograms. As with Katz’s holocamera, the projected real 
images in front of the holograms were scanned in x-y-z by a video camera mounted 
on computer-controlled micropositioning stages. Allowances for optical aberra- 
tions and refractive index mismatch between recording and replay were also made 
through selection of replay wavelength and beam angles (Hobson and Watson, 
2002). The HoloCam recorded high-quality images of calanoid copepods as well as 
of marine snow with both the in-line and off-line holographic cameras (Hobson 
and Watson, 2002). 

The collimated beam illumination of sample volume employed by both HoloCam 
and Holocamera gives unmagnified holograms of all particles present in the 
illuminated volume. The off-line method is capable of imaging objects, including 
their surface features, several millimeters in diameter. The particle density can be 
high since the reference wave does not pass through the sample and is thus not 
corrupted through scattering events. The method, however, gives poorer resolution 
for organisms below 500 pm and requires steps to minimize optical aberration on 
image reconstruction. Collimated beam in-line holography has allowed imaging of 
objects upward of 10 pm in length. For opaque objects, reconstruction will only 
give the silhouette of the objects and, since the reference wave passes through the 
imaged volume, the reference beam is also affected by high particle densities. 

The recorded holograms contain information about all the illuminated objects 
in the sample volume and images of objects are obtained through hologram 
reconstruction in selected image planes. To reconstruct larger objects, such as 
copepods, the density of reconstruction planes along the optic axis can be low and 
planes may be mm apart. To determine the density of plankton organisms in all 
size categories including picoplankton, however, requires a much higher density of 
reconstruction planes. To locate plankton particles 10 pm in diameter in a 300 mm 
long water column (liter size sample volume) would require a reconstruction 
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plane at least every 20-30 pm or a total of at least lO'* reconstruction planes. 
The information to be processed for a single hologram would be in the TB range. 
The quantitative evaluation of a sample thus requires enormous computing power 
and long reconstruction and image processing times. Even if the sampled volume 
is restricted to only 60 mm^, for example, the image information from a single 
hologram would still amount to 5 GB. The volume of information that must be 
processed is even more problematic when sequences of holograms are recorded 
to determine the trajectories and swimming behavior of plankton. It is therefore 
essential to develop fast reconstruction algorithms as well as automated methods for 
the evaluation of reconstructed holograms. A number of these automated methods 
have been described in the literature (Malkiel, Abras, and Katz, 2004; Hobson and 
Watson, 2002; Sun et al, 2008) and include automatic detection of in-focus objects 
in a reconstruction plane as well as automatic distortion corrections, false trace 
removal, and more. 

A much simplified and physically smaller system was recently developed by 
J. Watson’s group (Sun et al., 2007). Their eHoloCam, with a design depth of 
3 km, uses collimated beam illumination from a pulsed (1 mj per pulse at 4 ns 
pulse duration) frequency-doubled Nd-YAG laser at 532 nm. The design principle 
is similar to that shown in Figure 3.1b. Instead of a film recording medium, 
the eHoloGam uses a 2208 x 3000 pixel GMOS (complementary metal oxide 
semiconductor) detector with a square pixel pitch of 3.5 x 3.5 pm^. The camera can 
record holograms from a water column of 36.5 cm^ at a rate of 25 Hz. The camera 
was designed to be towed at speeds of up to 2 m s“^ and is self-contained in terms 
of power and data storage. 



3.4 

Point Source Digital In-Line Holographic Microscopy 

The recording systems discussed in Section 3.3 all used collimated laser beams, 
illuminated liter-size sample volumes, and were able to record microplankton. 
Most of the systems, however, were very bulky and had to use film to record the 
holograms. In addition, they were less successful in obtaining high-resolution 
images and trajectories of particles below 20 pm in diameter. Even for objects up 
to 50 pm in diameter (Malkiel, Alquaddoomi, and Katz, 1999), resolution was at 
times insufficient to decide between dinoflagellates, microzooplankton, or pieces 
of detritus. The aim of plankton imaging is to measure and monitor the abundance 
of living organisms in the oceans from microplankton down to picoplankton, 
including bacteria. The latter two are at the bottom of the food chain and serve 
as prey for ciliate plankton, which in turn are eaten by larger organisms such as 
copepods. Identification of organisms in the nano- and picoplankton size range, the 
determination of their contribution to the ocean biomass, and possibly their grazing 
rates are therefore important objectives for plankton imaging. The size range from 
0.5 to 10 pm can include bacteria, cyanobacteria, both auto- and heterotrophic 
picoflagellates, nanofiagellates, and also small dilates. The concentration of 
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planktonic microorganisms depends on location, temperature, and depth, and 
concentrations can show substantial diurnal as well as seasonal variations. 

We can obtain insight into plankton density and size distribution in ecologically 
significant areas by, for example, considering the data of Ferrier-Pages and Gattuso 
(1998). The concentration of heterotrophic bacteria in the water over a coral reef 
at Miyako Island (Japan) was measured to be 4-8 x 10^ cells ml~^ and represented 
a biomass of lOpgCG^. Autotrophic picoflagellates were present at a concentra- 
tion of 10^ cells mG^ (0.2pgCG^) while heterotrophic picoflagellates in the 2 pm 
size range amounted to 3-5 x 10^ cells mG^ (IpgCG^). Auto- and heterotrophic 
nanoflagellates in the 5-10 pm range were present at the 10^ cells mG^ (9 pg C G^) 
level while the concentration of microplankton was low. Diatoms (~40pm size 
range) , for example, were present at the 100 cells level and represented the lowest 
biomass. The size range from <lpm (10“*pgCcelD^) to a few tens of microm- 
eter (2 X lO^^pgCcelD^) was not accessible to past collimated beam holography 
systems. This size range is, however, important since it represents the high end of 
the marine plankton biomass and also determines production rates of organisms 
in the larger size ranges. To image plankton organisms and to identify them 
in their respective plankton groups requires true image resolution, and not just 
object visibility, at the micron level. This kind of resolution has been achieved with 
point source digital in-line holography (Garcia-Sucerquia et al., 2006). Examples 
of high-resolution point source DIHM images of algae and measurements of the 
temperature dependence of algae movement are given in Lewis et al. (2006). 

Figure 3.1c illustrates the principle of point source DIHM. A low-power laser in a 
pressure vessel is focused on a pinhole with the help of an objective. The pinhole is 
situated close to a thin optical window and the emerging light illuminates a conical 
region of the water to be examined. The detector is located in a second pressure 
chamber immediately behind a second optical window as shown in the figure. 

The pinhole emits nearly spherical waves, the reference waves, which interfere 
with waves scattered by objects to form a hologram at the detector. Hologram 
reconstruction then recovers images and coordinates of all objects in the illuminated 
volume. With modem GCD or GMOS cameras as detectors, this in-line digital 
holographic microscopy allows the construction of extremely small and robust 
high-resolution imaging systems. Use of a digital rather than film-based recording 
system was pioneered by Owen and Zozulya (2000) , who used a GG D array to capture 
and reconstruct in-line holograms of marine organisms to a depth of 16 feet. More 
recently the Dalhousie University holographic imaging group lead by H.J. Kreuzer 
and M.H. Jericho developed submersible point source DIHM instmments that 
achieved sub-micron resolution with GCD as well as CMOS detectors. 

3 . 4.1 

Instruments 

Two instruments have been developed. One for ocean studies with a design 
depth that covers most depths encountered over continental shelves. The second 
instrument was designed for shallower coastal waters and for limnological studies. 
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The design principle of the instruments is as shown in Figure 3.1c. The instru- 
ments consist of two chambers rigidly coupled together with four bolts. The source 
chamber houses the laser, microscope objective (lOOx), and pinhole while the 
recording chamber houses the CCD or CMOS camera. For shallow water applica- 
tions the holograms are transferred directly to the recording station via a FireWire 
while for the greater depth marine version of the instrument the camera chamber 
also housed the data transfer units for the USB-2 connection to the computer at 
the surface. Two USB-2 to ether net converters allowed data transmission up to 
480 Mbit s^'. The optical window on the camera chamber is 25 mm in diameter 
and 4 mm thick and is situated about 4 mm from the CCD chip. Construction 
of the source window is more problematic. For high-resolution, short pinhole to 
object distances are required. This implies that the source window has to be very 
thin. Since the window is close to the point source it, fortunately, also means that 
the window can be very small in diameter. The source windows therefore had 
diameters of 2 mm and were only 150 pm thick. The lasers used in our marine 
version of the microscope were solid-state lasers with wavelengths of 532 or 632 nm 
and powers between 3 and 15 mW, while our shallow water instrument used a 
25 mW, 405 nm wavelength laser. 

The CCD camera in the deep sea instrument is a Lumenera LulGOM with 
1392 X 1040 pixels of 6.4 pm in size. The camera has a frame rate of seven frames 
per second and a minimum exposure time of 0.045 ms. This short exposure time 
ensured that sharp images of microplankton in motion could be obtained. In 
an ocean environment with the submercible DIFIM suspended from a vessel, 
however, wave action and tidal currents generated excessive motion of the water 
in the imaging volume. Under these conditions and for a 15 mW laser, the typical 
exposure time of 0.2 ms was insufficient for sharp imaging of organisms. The 
imaging volume was therefore surrounded by an isolation chamber that could 
be opened to the ocean or closed for imaging. Further design details can be 
obtained from Jericho et al. (2006, 2010). We recently succeeded in performing 
high-resolution DIHM imaging with high numerical aperture optical fibers. The 
light intensity obtained with such fibers was sufficient to produce good holograms 
with 60 ps exposure times. Exposure times of a few microseconds on standard 
cameras will soon be available so that isolation mechanisms, as mentioned above, 
may become unnecessary in future. 

3.4.2 

Image Reconstruction 

The 3D structure of the object can be recovered from the digital two-dimensional 
hologram by applying a Kirchhoff- Helmholtz transform (Xu et al., 2002; Kreuzer, 
2002 ): 



In Eq. 3.1 the integration extends over the surface of the CCD chip with coordinates 
f = (X,Y,L), where L is the distance from the source to the center of the chip and 




(3.1) 
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I is the contrast image (hologram). The contrast image is obtained by subtracting 
the images with and without the object present or in the case of plankton imaging 
it is obtained by subtraction of successive exposures of a sample. The function 
K(r) is significantly structured and differs from zero only in the space region 
occupied by the object. By reconstructing the wave front K(r) on a number of 
planes at various distances from the source in the vicinity of the object, a 3D image 
can be built up from a single 2D hologram. The plot of |fC(r)|, on a 2D plane 
perpendicular to the optical axis, is equivalent to a single in-focus image taken 
in a conventional compound microscope. In DIHM one can generate a stack of 
2D holographic reconstructions from a single hologram. Combining such a stack 
into a 3D reconstruction will result in a 3D image of the object. For numerical 
implementation of the transform a fast algorithm was developed that evaluates K(r) 
without any approximations. It is now incorporated in a self-contained program 
packages, called Swordfish and Octopus, that not only perform the numerical 
reconstruction but also all data management and image visualization and analysis 
operations (Resolution Optics Inc.). 

In plankton imaging it is of considerable interest to not only record an image 
of an organism but also obtain a detailed record of its trajectory. This is easily 
achieved in DIHM with the procedure illustrated in Figure 3.2: (i) a sequence 
of holograms (h^) are recorded at fixed time intervals; (ii) undesired background 
effects, such as spurious interferences from optical windows, for example, are 
removed by subtracting consecutive hologram pairs, pixel by pixel, to generate 
difference holograms (hj — Iij), (h^ — h^), and so on; and (iii) the resulting difference 
holograms can be summed to generate a single data file that contains all holograms 
(with alternate signs), but that has the same size as any single original hologram. 
Reconstruction of this final data set with Eq. 3.1 in different depth planes then 



(a) A sequence of single 
holograms is recorded 2 
with the CCD camera 



!r 



(b) Create difference holograms 
by subtracting successive 
holograms pixel by pixel 



(c) Sum these difference 
holograms to obtain one final 
hologram 




Final hologram 



Difference 
hologram 19-20 



Figure 3.2 Recording and processing of a sequence of holograms to construct a sum holo- 
gram; subsequent reconstructions of the sum hologram in different slices through the sam- 
ple volume yields images of plankton trajectories. 
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gives a record of the trajectories of all objects in the illuminated sample volume 
{Xuetal, 2003). 

3.4.3 

Image Examples 

Figure 3.3 illustrates the high resolution that can be obtained with point source 
DIHM under laboratory conditions. Figure 3.3a is an image of a diatom obtained 
with a 0.5 pm pinhole, 405 nm illumination, and a 2000 x 2000 pixel camera. The 

1.3 pm wide ribs of the organism are well resolved. Figure 3.3b shows an image 
of a dividing E. coli bacterium that was imaged under similar conditions as the 
diatom. Although the septum is not clearly resolved, the partition region can be 
clearly identified and the image suggests sub-micron resolution. Figure 3.4 shows 
an image of a copepod that was obtained with our deep sea DIHM at a depth of 
10 m in coastal waters off Nova Scotia. The organism, in the microplanlcton size 
range, could be easily imaged with high resolution. Figure 3.4 shows the antennae 
and the numerous fibers that protrude from them. Small fibers on the side of the 
copepod (white arrow) have a diameter of 2 pm. Figure 3.5 shows a trajectory of 
a marine rotifer taken in coastal waters of Nova Scotia. Images were recorded at 
5 f s“^ but only every second frame was used for Figure 3.5. The rotifer undulates 
about the swimming direction and proceeds from right to left in Figure 3.5 at an 
average speed of 175 pm s^'. The recording of such particle trajectories can clearly 
help to distinguish living matter from detritus. The images demonstrate that in 




Figure 3.3 High-resolution DIHM image of (a) a diatom - the width of a rib-like struc- 
ture is 1.3 pm (source-camera distance = 1 5 mm; source-object distance = 500pm; 

= 405 nm; 0.5 pm pinhole); (b) Escherichia coli bacteria (source-camera dis- 
tance=15mm; source-object distance = 500 pm; = 405 nm; 0.5pm pinhole). 
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Figure 3.4 Image of a copepod taken 
in coastal waters of Nova Scotia at a 
depth of 10m. The hair-like protru- 
sions on the two antennae of the cope- 
pod are visible. The small fibers on the 



side of the copepod (white arrow) are 
2(im in diameter. (Source-camera dis- 
tance = 1 3.7 mm; source-object distance 
~700|im; X^^=A22nrx\\ 1.0[J.m pinhole.) 




Figure 3.5 Image sequence of a marine rotifer in motion; average swimming speed 
was 175|ims“^. (Source-camera distance = 1 3.7 mm; source-object distance = 600 (im; 

Ag^ = 422nm; l.Ojim pinhole.) 

situ imaging of marine plankton with resolution at the micron level is possible and 
that studies of nano- and even some picoplankton are possible with point source 
DIHM. Figure 3.6 shows a gallery of organisms that we managed to image with 
our submersible DIHM, while Figure 3.7 shows a 3D rendering of bacteria from 
an arctic lake on Axel Heiberg Island in the Canadian high arctic. 
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Figure 3.6 A gallery of marine and some 
freshwater organisms taken with the 
submersible DIHM and identified from 
the image: A: marine ciliate Favella; B: 
marine ciliate Strombidium conicum; C: 
dinoflagellate Ceratium sp.; D: diatom; E: 
rotifer; F: Paramecium; G: rotifer; H: ciliate 



Didinium; I: swimming ciliate; J: ciliate; K: 
marine pennate diatom; L and M: Pleu- 
rosigma /Gyrosigma sp.; N: marine silicoflag- 
ellate Dictyocha; O: centric diatom; and P: 
dinoflagellate (unknown Peridiniales); Q: 
unidentified organism. 



3.4.4 

Resolution 

The resolution of a microscope is given in terms of the numerical aperture (NA) 
of the objective lens. The lateral resolution, A, is given by Garcia-Sucerquia et al. 
(2006): 



At > X/2NA 
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Figure 3.7 Three-dimensional rendering of bacteria from an arctic lake on Axel Heiberg 
Island. (Source-camera distance = 21 mm; source-object distance = 800 jim; A = 405nm; 

0.5 |im pinhole; CCD chip size 1 .28 x 1 .28 cm^; reconstruction volume 500 x 500 x 300|im^.) 

The resolution along the optic axis is generally lower than the lateral resolution and 
is given by: 

Ai > ^/2(NA)^ 

The numerical aperture depends on the ratio of the detector width to the 
object-detector distance. The resolution thus increases at shorter wavelength 
and with larger screen width (larger NA). When the optic path crosses several 
media with different refractive indexes it is possible to show (Jericho et al., 2006) 
that hologram formation depends on an effective wavelength that is determined 
by the length of path through each medium. In our submersible instruments 
for high-resolution plankton imaging, the optic path involved air (n=l), water 
(n = 1.33), and glass (n = 1.52), and with a laser wavelength of 532 nm the effective 
wavelength was 2.j,g.= 422nm. The resolution of a holographic microscope can 
also be discussed in terms of the number of interference fringes that can be 
resolved by the recording medium. For digital CCD or CMOS detectors, faithful 
recording of interference fringes is only possible if the pixel size of the recording 
chip is much smaller than the smallest fringe spacing that needs to be recorded. 
Simulations with computer-generated holograms suggested that two scattering 
centers can be resolved if the number of captured fringes is large (tens of fringes) 
and that only fringes separated by more than three camera pixels contribute to 
image reconstruction and resolution. Either too few fringes or fringes that are too 
close together will lead to reduced resolution. In general, a shorter wavelength 
will produce more fringes, and usually results in higher resolution. However, the 
larger number of fringes is spread over a fixed number of pixels on the chip so 
that at shorter wavelength the average number of pixels per fringe decreases and 
the resolution may thus be less than expected. A more detailed discussion on 



62 I 3 Holographic Microscopy of Marine Organisms 

fringe number and spacing and its effect on resolution can be found in Jericho 
et al. (2006). Point source DIHM with a nearly spherical reference wave produces 
a magnified hologram at the recording screen. This implies that the number of 
fringes captured, and hence the resolution, depends on the object-screen distance. 
Figure 3.8 shows a calculation of the obtainable resolution for our deep-sea sub- 
mersible DIHM with the source-screen distance set to 13.7 mm and an effective 
wavelength of 422 nm. A resolution of 1pm is easily obtainable with the object 
close to the point source and the smallest measurable separation of two scattering 
points increases nearly linearly with source-object distance. The imaged volume 
is essentially determined by the volume of a pyramid that has the CCD chip as a 
base and the point source as the apex. As shown in Figure 3.8a, a resolution of 
2 pm or better is obtainable up to a source-object distance of 1.5 mm in an image 
volume of 1 mm^. At a source- screen distance of 4 mm the expected resolution is 
6 pm or better in a volume of about 10 mm^. If the gap between the source and 
camera chamber is increased then the imaged volume will increase, albeit with 
some reduction in resolution. Figure 3.8b shows a plot of resolution and imaged 
volume for a 60 mm source to screen distance and for an effective wavelength 
of 400 nm. At a source-object distance of 40 mm in a volume of 650 mm^, the 
expected resolution is 10.5 pm or better. Point source DIHM with pinhole diam- 
eters between 0.5 and 1 pm have consistently imaged picoplankton with micron 
resolution and can identify organisms and can determine their number densities 
as well as their swimming characteristics. 




(a) 



Source-object distance (mm) (b) 

Figure 3.8 (a) Obtainable resolution 

for marine submersible DIHM at 13.7mm 
source-screen distance and with 
7.gj^=422nm as function of source-object 
distance; 1000 x 1000 pixel CCD camera with 

6 pm square pixels. The resolution ( ) and 

image volume ( ) are shown that give 



Source-object distance (mm) 

the indicated or better resolution at a spec- 
ified source-screen distance, (b) The same 
as for (a) but with the distance between 
point-source and camera increased to 60 mm 
to increase the volume of the imaged sam- 
ple; A.g^= 400 nm. 
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3.4.5 

Volume Imaging Challenges 

In DIHM a single hologram contains all the information about scattering objects 
within the illuminated volume. Reconstruction of the imaged objects is then 
performed by slicing the volume into individual reconstruction planes oriented 
perpendicular to the optic axis. The number of slices needed will depend on 
the size of the imaged objects and on the desired resolution. To image a 20 pm 
diameter nanoplankton organism with high resolution will require a minimum of 
five reconstruction planes to home in on the slice where the object displays the 
sharpest silhouette. To reconstruct all particles of that size that may be distributed 
over a z-depth of 5 mm will thus require 1250 reconstruction operations that have 
to be performed on that one hologram. If the reconstruction time for a 1024 x 1024 
pixel hologram is 240 ms (Vr of this for a 512 x 512 hologram) on a PC, in-focus 
images of all organisms that may be present will take a minimum of 5 min. 
Holograms 512 x 512 in size still give acceptable resolution and an imaged volume 
of 10 mm^ (obtained at a source-object distance of ~5 mm; see Figure 3.8a) could 
be reconstructed in less than 2 min. The equivalence of 1 cm^ of sea water with 
high resolution can be imaged through multiple sampling of 10 mm^ volume 
elements. The 100 required samples would take about 3.3 h of reconstruction time 
for 512 X 512 holograms. This time can be shortened considerably if holograms 
in a sequence are added pixel by pixel to form sum holograms as explained in 
Figure 3.2. Each sum hologram would then contain information about all objects 
that were imaged in the sequence. For example, if ten individual holograms are 
added to form a sum hologram then imaging of 1 cm^ volume would generate 
only ten sum holograms that need to be reconstructed. For 512 x 512 holograms 
at 60 ms per z slice, the reconstruction time would take less than 20 min. 

In practice the search for organisms in the imaged volume can be accelerated 
considerably by first locating organisms approximately with more widely spaced 
z slices, which is then followed for each located organism with a higher slice 
density for organism identification. At low particle density, this method allows 
the reconstruction of organisms in a 1 cm^ volume in much less than 1 h. The 
example nevertheless illustrates the large amount of information that has to 
be processed for picoplankton imaging and the importance of high hologram 
reconstruction speeds. We have recently obtained very fast hologram reconstruction 
with the Helmholtz -Kirchhoff transform using graphics processing units {GPUs) 
as coprocessors. With the Tesla C-1060 card, for example, we can perform a 
single reconstruction of a 2048 x 2048 hologram in 40 ms or reconstruction of a 
512 X 512 hologram in 7 ms (reconstruction time did not scale well with hologram 
size). Reconstruction of all pico- and nanoplankton objects in a cubic centimeter 
of water can now be achieved in a few minutes on a PC. The high reconstruction 
speed of nearly 150 reconstructions per second for a 512 x 512 pixel hologram 
implies a near video speed reconstruction capability and real-time holographic 
imaging of plankton motion through several hundred pm thick sample volumes is 
now also possible. In the next few years the new generation of GPUs is expected 
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to allow further increases in reconstruction speeds by perhaps as much as an 
order of magnitude. This substantial gain in reconstruction speed could then be 
increased further if several graphics processing cards are operated in parallel so 
that high-resolution real time 3D imaging of picoplankton will soon be possible. 



3.5 

Future Outlook 

For microorganisms above ~30pm identification can be accomplished through 
optical imaging or reconstructions of DIHM holograms. Organisms smaller than 
~20 pm such as algae, mammalian cells, many forms of nano- and picoplankton 
in marine environments, as well as organisms at the micron length scale, such 
as bacteria are, however, generally difficult to identify from their static images 
alone. However, organisms often have characteristic swimming behavior that 
can help in their identification. For example, consider the three bacteria Vibrio 
cholerae, Escherichia coli, and Enterococcus faecalis. All three are indicator microbes 
for the presence of harmful organisms in ship’s ballast water. The first is a 
curved-rod shaped bacterium with a single flagellum. The second is a straight rod 
bacterium with several flagella that bundle when the bacterium swims forward but 
periodically unbundle to give the bacterium a characteristic tumbling motion. The 
third organism is a non-motile bacterium that exists as pairs or short chains of 
individuals. The trajectories of a curved-rod bacterium differ from the straight and 
occasional tumbling motion of E. coli while the third organism should only execute 
Brownian motion steps. 

An important future objective therefore is to exploit the high depth of field of 
DIHM and to measure and characterize organism trajectories. At high organism 
density, trajectory analysis will have to be automated and powerful analysis software 
needs to be developed so that organism coordinates, speed, swimming direction, 
trajectory curvature, and general trajectory shape (i.e., straight, circular, helical, 
etc.) can be determined and cataloged. 

Important progress in that direction has already been made by Heydt et al. 
(2009) In connection with a general research program on surface fouling by 
marine organisms by Rosenhahn et al. (2010) the authors used point source 
DIHM to study the three-dimensional settling dynamics of algal zoospores in 
the vicinity of surfaces. They developed algorithms and techniques for high- 
throughput analysis of holographic microscopy data. The large depth of field of 
holographic microscopy enabled them to determine, in a nearly automatic manner, 
distributions of swimming speeds as well as diving directions of multiple zoospores 
near a surface (Heydt et al., 2007, 2009). 

The high resolution obtainable with point source DIHM together with the 
extreme simplicity of DIHM construction makes these instruments excellent 
candidates for the exploration of microscopic life forms in harsh and challenging 
environments. Their potential for monitoring plankton populations of coastal 
waters has been well demonstrated, as discussed in this chapter. Furthermore, 
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DIHM technology in future can include automatic in situ monitoring of plankton 
populations by instruments powered by photovoltaics and that transmit hologram 
image information to laboratories via satellite links. More challenging, but also 
within present technological capability, is the design of instruments that can image 
microscopic life in deep-sea trenches or even carry out exobiological studies on 
other planets (Jericho et al., 2010). At an ocean depth of 10km, instruments 
have to be completely self-contained with no power and data transfer links to a 
surface vessel and all power supplies, imaging protocols, and data manipulation 
and storage must be controlled by on-board computers. For imaging at extreme 
depth, the camera and point source chambers require thick walls. This makes it 
impossible to position the pin hole of the point source close to the source window as 
required for high-resolution imaging. In collaboration with Alexander Bochdansky 
from the Old Dominion University, Norfolk VA we recently developed an optic 
fiber based point source that allowed high-resolution imaging to a depth of 6000 m 
(Bochdansky et al. 2013). Very little is known about the existence and behavioral 
traits of microscopic organisms at that depth and pressure and DIHM can help in 
providing a window on this last frontier on our planet. 
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4 

Confocal Laser Scanning Microscopy - Detailed 
Three-Dimensional Morphological Imaging of Marine 
Organisms 

Jan Michels 



4.1 

Introduction 

Although the principle of confocal laser scanning microscopy {CLSM) had been 
invented in 1955 (Minsky, 1988), it was not until about three decades later that 
new scanning techniques and developments in the fields of microelectronics and 
laser and computer technologies enabled the construction of the first commercial 
confocal laser scanning microscope (e.g., Amos and White, 2003). Since then 
CLSM has become a powerful standard technique in the field of life sciences, 
and numerous very successful CLSM methods have been developed for high- 
resolution visualizations and analyses of a large variety of tissues, compounds, 
and substances in biological and medical specimens. In recent years the greatly 
increased power of computer and data processing systems in combination with the 
development of faster scan techniques and new effective CLSM and 3D software 
has resulted in CLSM now also having a large potential for 3D and live cell imaging. 
Today, CLSM is mainly used in the scientific disciplines cell biology, molecular 
biology, biochemistry, and medical science. In the context of some applications 
in materials science this technique has also gained importance within the last 
years. By contrast, the use of CLSM in classical disciplines of biology such as 
morphology and taxonomy is still rare. The present chapter describes the technical 
and methodological background of CLSM, highlights, based on various examples, 
the potential of CLSM for precise morphological imaging of marine organisms, 
and gives information about the most important aspects of the respective methods. 



4.2 

Technical and Methodological Aspects of Confocal Laser Scanning Microscopy 

When visualizing thick specimens with conventional light microscopy techniques, 
such as bright field microscopy and wide-field fluorescence microscopy, the struc- 
tures, which are in focus, typically do not appear clear and sharp but look blurry 
instead. The reason for this is that during the visualization process light coming 
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from all layers of the specimen reaches the oculars of the microscope or the sensor 
of the camera. However, only the light signal coming from a specific layer at the 
focal plane provides sharp and clear information about the respective structures. 
The thickness of this layer, called the focal layer in the following, is equal to the 
microscope optics’ depth of field, which is limited and typically rather small at high 
magnifications. The light signal coming from this focal layer is overlaid with light 
information originating from out-of-focus layers above and below the focal layer. 
Because of this overlay the structures of interest, which are in focus, appear blurry. 
By contrast, in confocal microscopy, as the name indicates, only light from the focal 
layer is used to create the micrographs while out-of-focus light either does not occur 
or its detection is avoided. Several different confocal microscopy techniques have 
been developed, all of them using different ways to achieve the goal of exclusively 
visualizing structures located in the focal layer. In CLSM, this is achieved by a 
specific pinhole technique. The technical details of CLSM are described below: 

1) Laser light is used to illuminate the specimen. This light passes a first 
(illumination) pinhole (Figure T.la.a) [Modern commercial CLSM systems do 
no longer feature such a pinhole because it is not essential (Figure 4.1a.b).], 
enters the optical system via a dichroic mirror, and is focused by the optical 
system, resulting in a tiny illumination spot in the focal plane within the 
specimen (Figure 4.1a). 

2) Light coming from the specimen passes the optical system in the opposite 
direction and is focused on the second side of the optical system, opposite 
to the specimen. On this side, a second (exit or detection) pinhole is located 
exactly in the second focal (micrograph) plane, which is optically conjugated to 
the first focal plane in the specimen. Owing to this setup, only light coming 
from a distinct layer at the first focal plane can pass the pinhole (Figure 4.1b), 
while nearly all light coming from all other layers of the specimen is blocked 
(or filtered out) by the pinhole (Figure 4.1c,d). 

3) Behind the detection pinhole a specific detector collects and processes the light. 
By means of a photomultiplier the light signal is transformed into an electronic 
signal, whose intensity can be increased by applying a certain detector gain. 

4) The light detected can be laser light reflected by the specimen, fluorescence 
light exhibited by the specimen, or a combination of both. For fluorescence 
visualizations it has to be ensured that the reflected laser light is filtered out and 
the fluorescence of interest is detected, which is achieved by specific detection 
setups. While in conventional CLSM systems classical fluorescence emission 
filters have been used, most of the recent commercial systems are equipped 
with spectral detectors, which enable the user to select any desired detection 
wavelength range. 

In context with this setup, several aspects have to be considered. The thickness 
of the layer, from which light is detected, depends on the size of the pinhole. The 
smaller the pinhole the thinner this layer is and vice versa (Figure 4.2). Only if 
the thickness of this layer (called the optical section in the following) is equal to or 
smaller than the thickness of the focal layer (depth of field) of the optical system, 
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Figure 4.1 Schematic illustration of the 
CLSM principle. The excitation light path is 
shown in green (a). Three exemplary paths 
of either fluorescence light originating in 
the specimen or laser light reflected by the 
specimen are shown in red and represent 



light coming from the focal layer (b) and 
from two out-of-focus layers (c, d). While 
older CLSM systems feature an illumination 
pinhole (a.a), the illumination techniques 
of modern CLSM systems are normally no 
longer equipped with such a pinhole (a.b). 
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Figure 4.2 Schematic illustration of the 
influence of the pinhole size on the thick- 
ness of the optical sections (for convenience 
only one lens is shown). A pinhole with a 
given size lets light pass to the detector. 

This light comes from an optical section 



with a defined thickness (a). If the pinhole 
size decreases light coming from upper and 
lower parts of the optical section will be 
blocked by the pinhole (b). As a result, only 
light coming from a thinner optical section 
will reach the detector (c). 



the microscope setup will be in a confocal state. The thickness of the focal layer 
depends on the properties of the microscope optics. Accordingly, the maximum 
size of the pinhole, at which the system is still in a confocal state, also depends on 
these properties. 

If the microscope system has a tiny punctiform illumination its resolution will 
depend on the numerical aperture of the objective used and the wavelength of 
the light detected and will be limited by diffraction. In such a microscope system 
an infinitely small luminous object point is visualized as a circular so-called Airy 
diffraction pattern featuring a central bright disk, the Airy disk, and progressively 
weaker concentric dark and bright rings (Inoue, 2006). The radius of the Airy 
disk is determined by: 



'■"■'“"■“na 



where NA is the numerical aperture of the objective used and k is the wavelength 
of the light detected (Inoue, 2006). 

The diffraction patterns of two incoherently radiating light points, which are 
located in the focal plane and separated by a certain distance, are visualized next 
to each other in the micrograph plane. According to the Rayleigh criterion these 
light points, if they are equally bright, will only be fully resolved by the microscope 
system and visible on the micrograph if the distance between them is larger or 
equal to the radius of the Airy disk (Inoue, 2006). In CLSM, the size of the image of 
the Airy disk displayed by the optical system in the pinhole plane (depending on the 
numerical aperture of the objective applied, the wavelength of the light detected, 
and the magnification of the optical system) is used to describe the opening size 
of the pinhole. In this context, 1 Airy unit (AU) corresponds to the size of the 
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Airy disk image. The maximum size of the pinhole, at which the system is still a 
confocal one, is about 1.2 AU. 



4.3 

Prerequisites for Generating High-Quality Confocal Laser Scanning Micrographs 

To create micrographs of the optical sections the laser illumination spot is moved 
line by line over the field of view with a certain scan velocity. At the same time the 
light coming from the single areas illuminated is detected, and the micrograph is 
generated line by line. A zoom function enables the user to freely choose the size 
of the scanned area (field of view) within certain ranges, and the micrograph size 
can also be freely chosen (in modern commercial CLSM systems up to a size of 
8192 X 8192 pixels). In this context, specific requirements have to be fulfilled to 
obtain micrographs showing the visualized structures with the highest resolution 
possible. The Nyquist- Shannon sampling theorem says that for a precise and 
complete conversion of a continuous analog signal into a digital signal it is 
necessary to sample the analog signal with a frequency, which is at least two times 
the frequency of the analog signal [In practice, often higher minimum sampling 
frequencies are recommended (Centonze and Pawley, 2006).]. Accordingly, for 
the creation of optical sections, which show all structural information resolved by 
the objective, the pixel size, which is the real size each pixel of the micrograph 
represents, must not be larger than half the size of the smallest structure resolved 
by the objective. That means, for example, when applying an objective with a 
resolution of 200 nm, that the micrograph must contain at least as many pixels as 
result in each pixel representing a size of 100 nm. 

As mentioned above, if the pinhole is closed to a size of about 1.2 AU the setup will 
be a confocal one. However, it is advisable to apply smaller pinhole sizes to obtain 
thinner optical sections resulting in clearer and crisper micrographs. Another 
positive effect of further decreasing the pinhole size is that the xy-resolution of 
the confocal system increases above the resolution described above (Centonze and 
Pawley, 2006). On the other hand, the amount of light being detected decreases 
significantly with a decreasing pinhole size. For this reason such a setting will 
only be advantageous if light with high intensities (either reflected laser light 
or intense fluorescence light) coming from the specimens is detected, while the 
visualization of specimens exhibiting relatively low fluorescence intensities would 
demand either intense excitation laser light or high gain values to increase the 
signal intensity. This would result in an increased risk of fluorescence bleaching 
or a decreased signal-to-noise ratio. Accordingly, for a large proportion of CLSM 
applications in practice the use of a pinhole size of 1 AU is recommended as a good 
compromise. 

In terms of the laser light intensity applied, the selected scan velocity, and the 
detection of the light signal a few aspects have to be kept in mind. Both the 
scan velocity and the detector gain are positively correlated with the signal and 
micrograph noise. This means that increased scan velocity and gain will result in 
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an increased noise (or a decreased signal-to-noise ratio) being detected and visible 
on the micrographs. In general, it is recommended that slow scan velocities and 
low gain values are used, the latter being achieved by higher laser intensities in 
case of low fluorescence intensities. However, the specimens often do not allow for 
this. For example, unstable or moving specimens (as often present, for example, in 
the framework of live cell imaging) and fluorescence dyes, which are very sensitive 
to bleaching, preclude long scanning times and high laser intensities, respectively. 
Accordingly, in such situations good compromises have to be found. Modern 
detection techniques have greatly improved and feature relatively low noise levels, 
allowing for the application of relatively fast scan velocities and high gain values. 
Despite all of this, sometimes it is not possible to obtain high-quality micrographs 
because of the properties of the specimens and the fluorescence dyes used. In 
such situations reducing the micrograph size (and thus increasing the pixel size) 
helow (above) the one necessary to meet the Nyquist- Shannon sampling theorem 
is often a good option to obtain reasonable results, which can still provide all the 
information desired. For many scientific questions it is not essential to approximate 
the resolution limit of the CLSM system, while other requirements are important 
and can also be matched with smaller micrograph sizes. 

The optical sectioning properties of CLSM make 3D analyses of specimens 
possible. For this purpose, micrographs of a defined number of optical sections, 
which are uniformly distributed along the z-axis and cover the whole structures of 
interest, are created (Figure 4.3). The resulting micrograph stack can he processed 
with methods identical to those performed with 3D data obtained from other 3D 
imaging techniques such as computer tomography. A very common method for 
displaying 3D data is the maximum intensity projection (MIP), a volume rendering 
method (Figure 4.3a). To create this pseudo-3D visualization the voxels (volumetric 
picture elements, each representing a value on a regular grid in a three-dimensional 
space) with the maximum intensities along parallel axes from the viewpoint to the 
projection plane are projected in the visualization plane. MIPs provide a good 3D 
impression of the morphology of the analyzed structures, which is adequate for 
many applications. If it is necessary to perform precise measurements, calculations, 
or simulations with the 3D data, different types of 3D models can be created (e.g.. 
Figure 4.3b-f). 

As for the micrograph size (number of pixels in the xy-direction) of the optical 
sections, it is also important to meet the Nyquist- Shannon sampling theorem 
when collecting data in the z-direction. To achieve maximum resolution in the 
z-direction when creating a micrograph stack with CLSM, the distance between the 
centers (focal planes) of two consecutive optical sections must be at least half the 
thickness of the respective optical sections. Again, to decrease the scanning time, 
this distance can be increased to reduce the number of micrographs per stack. 
However, this should only be done if it is not necessary to create a precise 3D 
model afterwards. For the creation of MIPs it is often not problematic to reduce the 
number of micrographs below that necessary according to the Nyquist- Shannon 
sampling theorem. In any case this has to be done carefully as such a reduced 
micrograph number can sometimes cause artifacts also in MIPs. 
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Figure 4.3 Part of a CLSM z-stack and 
examples of pseudo-3D and 3D visual- 
izations created from this z-stack. The 15 
small micrographs in the upper section 
of the figure are a part of a z-stack con- 
sisting of 1 79 optical sections covering a 



complete mandibular gnathobase of the 
marine cyclopoid copepod Acanthocyclops 
mirnyi. (a) Maximum intensity projection, (b) 
isosurface model, and (c-f) smoothed iso- 
surface model shown from different angles. 
Scale bar = 20 pm. 



With respect to the signal intensities and micrograph qualities, it is also important 
to take care of the embedding and immersion media. Usually, several components 
(e.g., tissue of the specimen, embedding medium, cover slip, immersion medium, 
glass of the objective) form the complete optical system. In general, if light passes 
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the border between different materials or substances with different refractive 
indices reflection and refraction will take place. The larger the differences between 
the refractive indices the more pronounced these effects are. Accordingly, to 
achieve good signal intensities and micrograph qualities it is important to choose 
embedding and immersion media and cover slips whose refractive indices are 
as close as possible to the refractive index of the glass of the objective lenses. 
Often, immersion oil with a refractive index identical to that of the glass is used 
as immersion medium, and several potential embedding media with the same 
refractive index have been developed so far. However, in many situations the 
properties of the preparations demand specific embedding media (e.g., water or 
certain buffers for the visualization of live specimens) with other refractive indices 
resulting in decreased signal and micrograph qualities. When precise and reliable 
3D analyses are desired it is very important to avoid mismatches between the 
refractive indices of the embedding and the immersion media (Egner and Hell, 
2006). If such a mismatch exists the movement of the objective with constant 
intervals along the z-axis will result in increasing or decreasing (depending on 
the ratio of the refractive indices) z-axis intervals between the respective focal 
planes, causing axial distortion in the 3D data set. For exactly known refractive 
indices this effect can be mathematically determined and removed from the data 
set after the micrograph acquisition. However, it is generally recommended that 
this effect is avoided by using embedding and immersion media having identical 
refractive indices. As a negative result of this, in situations in which the application 
of embedding media with relatively low refractive indices (e.g., water) is essential, 
losses in the signal and micrograph qualities have to be accepted. But in most cases 
the latter effect is negligible and compensated for by the clearly improved quality 
of the 3D information. 

In the framework of analyses of internal structures the thickness and the 
transparency of the preparations have a considerable impact on the CLSM results. 
If the preparations are too thick or not transparent enough (or a combination of 
both) refraction and absorption effects will result in poor quality signals coming 
from the deeper layers in the preparation part located opposite the lasers. In 
practice, good signal qualities and good signal yield are often limited to the upper 
100-200 pm of the preparations. 

As mentioned above, both laser light reflected by the specimen and fluorescence 
light originating in the specimen can be used for CLSM visualizations. In the 
following sections respective visualizations of the morphology of marine organisms 
will be demonstrated using selected examples. 



4.4 

Using Autofluorescences for Detailed Three-Dimensional Morphological Imaging 

A considerable variety of substances and materials present in a large number 
of organisms exhibits autofluorescences. In the context of many fluorescence 
microscopy applications such autofluorescences can interfere with the results 
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and are, therefore, unwanted. However, at the same time, if they are sufficiently 
intensive they can be successfully applied for effective microscopic analyses. 

In general it is advisable to test all species of interest for autofluorescences at the 
start of the analyses. To increase the chance of finding such autofluorescences all 
laser lines available should be applied successively, and it should be ensured that 
each of the excited autofluorescences is completely visualized by using either appro- 
priate long-pass filters or, in case of a spectral detection system, a broad detection 
range. Such settings are often very appropriate for visualizations of complete speci- 
mens. However, if specimens exhibit different intensive autofluorescences it might 
be of interest to visualize a combination of these or a combination of autofluores- 
cences and the fluorescences of dyes (see below). For such situations it is necessary 
to gain information about the spectral properties of the autofluorescences, and 
then the detection can be refined by adjusting the selection of the emission filters 
or the settings of the spectral detection system to the autofluorescence properties. 
Spectral detection systems enable the user to perform so-called lambda scans, 
which provide detailed information about the spectral properties of fluorescences. 
With the results it is possible to specifically detect the autofluorescences and the 
fluorescences of dyes. 

Only a few autofluorescences have been investigated in detail so far. Among 
those present in marine organisms, the chlorophyll autofluorescences are well 
known and widely applied to visualize the chloroplasts of algae cells. Typically, the 
excitation and emission maxima of the combined autofluorescences of different 
algae chlorophylls are located in the blue and red parts of the light spectrum, 
respectively. They can easily be detected in both live and dead cells (Figures 4.4a,c,d,f 
and 4.6b, d,e below). Besides the chlorophylls, algae contain additional pigments, 
some of them being specific for single algae taxa and having autofluorescence 
properties different from those of the chlorophylls. Because the excitation and 
emission spectra of the chlorophylls and of many other important pigments are 
known in detail (e.g., French and Young, 1952; French et al, 1956) different algae 
taxa can be visualized separately by choosing appropriate excitation wavelengths 
and emission detection ranges. 

Among the arthropods, the exoskeletons of many taxa also exhibit intensive 
autofluorescences. In recent years several studies have demonstrated that these 
autofluorescences allow for precise CLSM visualizations (e.g., Zill et al., 2000; 
Klaus, Kulasekera, and Schawaroch, 2003; Schawaroch, Grimaldi, and Klaus, 2005; 
Klaus and Schawaroch, 2006; Michels, 2007; Michels and Gorb, 2012). In the 
field of marine biology, the application of the respective methods can contribute 
significantly to morphological analyses of crustaceans, which play important roles 
in many marine ecosystems and food webs and are, therefore, the subject of a 
large variety of studies. Even relatively thin exoskeletons such as those often found 
in, for example, calanoid and cyclopoid copepods can be efficiently visualized 
based on autofluorescences, resulting in precise 3D models of the structures 
of interest (Figure 4.3). However, an occasional problem is the occurrence of 
pronounced differences in the autofluorescence intensities between different parts 
of the exoskeletons caused by differences in the thickness and the density of the 
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Figure 4.4 Confocal laser scanning micro- 
graphs showing frustules and chloroplasts 
of the marine diatom Coscinodiscus waiiesii. 
(a-c, e-h) MIPs, (d) optical section, (a, d, 
f) Overlays of the autofluorescence of the 
chloroplasts (red) and the PDMPO fluores- 
cence (excited with 405 nm) of the stained 
frustules (green) of living diatoms, (b) 
Frustule from (a) illustrated separately, (c) 
Chloroplasts from (a) illustrated separately. 



(d) Exposure to laser light with too high 
intensities stressed the diatom cell, which is 
indicated by the abnormal distribution of the 
chloroplasts. (e, g, h) Isolated and cleaned 
frustules stained in living diatoms with the 
fluorescence dye rhodamine 19. The fluores- 
cence of rhodamine 19 is shown in the glow 
mode of the software Leica LAS AF. Scale 
bars = (a-f) 50, (g) 10, and (h) 5 p.m. 



respective exoskeleton material. In such situations, sometimes oversaturation of 
signals coming from single parts of the specimens has to be accepted to successfully 
visualize the whole specimens (see, for example, Figure 4.5a,b). 

The maximum resolution of CLSM is much lower than that of scanning electron 
microscopy (SEM), but at relatively low magnifications the results obtained with 
both techniques are comparable in terms of structure detail information (see 
below). Compared to SEM, CLSM visualizations using autofluorescences have two 
significant advantages: (i) Most of the preparation methods are much easier and 
much faster. For example, in case of CLSM analyses of whole specimens the only 
working step is to transfer these specimens to the embedding medium and place 
a cover slip on top of them, (ii) The specimens can be visualized in a non-invasive 
and non-damaging way. For a large variety of applications glycerin or a mixture 
of glycerin and water have proved to be very appropriate (e.g., Michels, 2007; 
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Figure 4.5 CLSM MIPs showing overlays 
of different autofluorescences present in 
the exoskeleton of marine copepods. (a) 
Ventral view of the harpacticoid copepod 
Alteutha potter, (b) Pair of fifth swimming 



legs of a male calanoid copepod Heterorhab- 
dus sp. (c) Dorsal view of the harpacticoid 
copepod Ceratonotus steiningeri. (d) Mandibu- 
lar gnathobase of Heterorhabdus sp. Scale 
bars = (a) 200 and (b-d) lOOpm. 



Michels and Gorb, 2012). The application of such embedding media does not 
harm the specimens and allows for removing and cleaning them from the medium 
after CLSM visualization. This enables the user to include the same specimens in 
other analyses afterwards. Furthermore, rare or unique museum specimens can 
be analyzed and returned without any alteration. 

In addition to morphological imaging, the autofluorescences also permit analyses 
of material compositions (Michels, 2007; Michels and Gorb, 2012). However, such 
analyses have to be performed with great care because the sources and the properties 
of many autofluorescences are not yet known. For example, the autofluorescences 
observed in arthropod exoskeletons have, with one exception, not yet been analyzed 
in an absolutely certain way. Only the autofluorescence properties of resilin, a 
very elastic and flexible protein found in many arthropod exoskeleton structures 






80 I 4 Confocal Laser Scanning Microscopy 

(e.g., Weis-Fogh, 1960; Andersen and Weis-Fogh, 1964), are known in detail (e.g., 
Andersen, 1963). Accordingly, the presence of resilin can be reliably determined 
by specifically visualizing its typical autofluorescence (shown below in Figures 
4.7a and 4.8a). In contrast, for the sources of other autofluorescences found 
in arthropod exoskeletons only a few indications are known. It seems that the 
autofluorescences of strongly sclerotized chitinous materials are mainly excited 
by green to red light, with the emission being mainly in the red part of the 
light spectrum, while less sclerotized or non-sclerotized chitinous materials exhibit 
mainly green autofluorescences with excitation maxima located in the blue to green 
part of the light spectrum (Michels and Gorb, 2012). If different autofluorescences 
are detected separately and the resulting fluorescence micrographs are overlaid 
differences in the autofluorescence composition will indicate differences in the 
material composition (e.g.. Figure 4.5c,d). In the case of such analyses it is essential 
that for each of the autofluorescences the signal is detected with maximum intensity 
while simultaneously any oversaturation is avoided. Otherwise, the visualization of 
proportions of the different materials would bear the risk of not being in accordance 
with the real material composition (e.g.. Figure 4.5a,b). In addition, to obtain reliable 
results, for such analyses only fresh specimens should be used because to date the 
influence of fixation chemicals on the properties of the autofluorescences has not 
been investigated. 

4.5 

Application of Fluorescence Dyes 

In the framework of many studies and analyses the autofluorescences present 
in organisms are not intensive enough for successful visualizations. In such 
situations, fluorescence dyes, which today exist in a great diversity (Tsien, Ernst, 
and Waggoner, 2006; Johnson, 2006), can be used instead. This allows for selective 
fluorescence labeling of structures, tissues, cell compartments, substances, and 
physiological processes of interest. In the following, some examples are described of 
the application of fluorescence dyes in combination with CLSM for morphological 
analyses of marine organisms. 

Diatoms dominate the phytoplankton in many ocean areas and contribute sig- 
nificantly to biogeochemical processes in marine ecosystems. For this reason 
it is of great interest to intensively study a wide variety of different aspects 
of diatom biology and physiology. For numerous geological, biological, and 
biomechanical investigations detailed knowledge of diatom frustule formation, 
morphology, and architecture is of importance. In recent years several studies 
have demonstrated the great potential of the fluorescent dye 2-(4-pyridyl)-5-((4- 
(2-dimethylaminoethylaminocarbamoyl)methoxy)-phenyl)oxazole (PDMPO) for a 
very efficient and quick incorporation into the newly synthesized frustules of live 
diatoms (Shimizu et al, 2001; Hazelaar et al, 2005; Leblanc and Hutchins, 2005). 
In the forming frustules PDMPO is deposited together with freshly deposited silica, 
resulting in the frustules exhibiting a typical intense green fluorescence (Shimizu 
et al, 2001). This allows for both live cell imaging of diatoms including their 
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frustules (Figure 4.4a,b,d,f) and visualizing diatom fmstules separated and cleaned 
from the organic material after the staining procedure. In general, PDMPO can be 
successfully applied to stain all newly formed structures containing biogenic silica 
such as siliceous skeletons of radiolarians (Ogane et al., 2010). However, one disad- 
vantage of this dye is that its excitation maximum is located in the UV range of the 
light spectrum. Today, not many commercial CLSM systems present in institutes 
and laboratories are equipped with UV lasers. In addition, lasers with a wave- 
length of 405 nm, which also excites the PDMPO fluorescence (Figure 4.4a,b,d,f), 
are still infrequent. Accordingly, not all CLSM users have the opportunity to use 
PDMPO. However, several other dyes, including different rhodamine derivatives, 
have been proven to efficiently label the silica of newly forming frustules of live 
diatoms, and the fluorescences of all of these dyes are excited with laser light 
wavelengths typically available in most CLSM systems (Figure 4.4e,g,h; Li, Chu, 
and Lee, 1989; Kucki et al., 2006; Descles et al., 2008; Annenkov et al., 2010; Kucki 
and Fuhrmann-Lieker, 2012). 

In all the studies mentioned above, live diatoms have been used and the 
dyes’ potential to stain newly forming frustules has been evaluated. However, 
in many situations it might be of interest to stain the frustules of dead diatom 
cells or isolated diatom frustules (e.g., from sediment samples). The potential of 
two rhodamine derivatives to stain readily polymerized silica of diatom frustules 
was reported some years ago, but no details were described (Hildebrand and 
Palenik, 2003). Recent studies showed that conjugates of silane coupling agents 
such as (3-aminopropyl)trimethoxysilane and fluorescence dyes (e.g., fluorescein 
isothiocyanate) efficiently label the silica of isolated diatom frustules and allow 
for precise CLSM visualizations and 3D modeling (Friedrichs, Maier, and Hamm, 
2012; Friedrichs, 2013). Furthermore, in an analysis of the material composition 
of copepod gnathobases a respective staining method was successfully applied to 
stain silica (Michels, Vogt, and Gorb, 2012), indicating that such conjugates label 
all structures containing biogenic silica. 

Besides diatoms, dinoflagellates represent another important group of marine 
unicellular algae. Among them, armored forms feature a so-called theca consisting 
of cellulose plates, which can be labeled with the fluorescence dye Fluorescent 
Brightener 28 (synonym: Calcofluor White M2R) (Fritz and Triemer, 1985), a dye 
known to stain, among other compounds, cellulose and chitin in cell walls (e.g., 
Maeda and Ishida, 1967; Hughes and McCully, 1975; Herth and Schnepf, 1980). 

The staining of the dinoflagellate cellulose plates is very efficient (Figure 4.6a-c). 

In general. Fluorescent Brightener 28 seems to allow for live cell imaging, because 
concentrations of 10-20 pg mL“^ within the growth media are reported to be non- 
toxic for the cells (Fritz and Triemer, 1985). However, the thecae of living motile 
cells are not stained, which is probably due to the intact outermost membrane 
preventing contact between the dye and the cellulose plates (Fritz and Triemer, 
1985). On the other hand. Fluorescent Brightener 28 stains the cellulose structures 
of living cysts and those of motile cells with damaged or removed outer membranes 
enabling the dye to bind to the cellulose (Fritz and Triemer, 1985). Unfortunately, 
to damage the membranes the cells must be treated harshly, which often causes the 
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Figure 4.6 CLSM MIPs showing thecae 
and chloroplasts of living dinoflagellates 
of the marine genus Ceratium. The thecae 
were stained with Fluorescent Brightener 28 
whose fluorescence was excited with 405 nm 
and is shown in gray, (a) Detailed view of a 
theca, (b) Overlay of the Fluorescent Bright- 
ener 28 fluorescence of a stained theca and 
the autofluorescence of the chloroplasts 



(red), (c) Theca from (b) illustrated sepa- 
rately. (d) Chloroplasts from (b) illustrated 
separately. After the staining, the cell was 
still alive. However, compared to the chloro- 
plasts of an unstained Ceratium sp. (e), the 
chloroplasts of the stained Ceratium sp. (d) 
looked slightly degenerated, indicating that 
the cell had been harmed by the staining 
procedure. Scale bars = 25 pm. 



cells’ death and leads to alteration of intracellular structures such as chloroplasts 
(Figure 4.6h,d). Accordingly, live cell imaging of motile dinoflagellates is very 
difficult. By contrast, the analyses of fixed cells (e.g., with glutardialdehyde) yield 
good results (Fritz and Triemer, 1985). Like PDMPO, Fluorescent Brightener 28 
has the disadvantage that its excitation maximum is located in the UV range of the 
light spectrum. The fluorescence of Fluorescent Brightener 28 can also he excited 
with 405 nm laser light (Figure 4.6a-c), hut this is not as effective as excitation with 
UV light. For this reason it might be of interest to use other dyes. In this context, a 
potential dye might be Congo red, because it has been reported to efficiently stain 
cellulose fibers in plant cell walls (Frey-Wyssling, 1976). Unfortunately, up to now 
its potential to stain the thecae of dinoflagellates has not been analyzed. 

As mentioned above, marine crustaceans are extensively studied because they 
play important roles in marine ecosystems. The exoskeleton autofluorescences of 
many species are relatively weak and, therefore, do not allow for successful CLSM 
visualizations of the external morphologies. In such situations, there are different 
options to solve this problem. In some studies complete specimens have been 
stained with non-specific or not completely specific fluorescence dyes, for example 
with Dil, a lipophilic membrane dye, which stains cell membranes and enables the 
user to visualize whole organisms (e.g., Carotenuto, 1999; Zupo and Buttino, 2001; 
Buttino et al., 2003). However, in the case of arthropods this is not efficient, because 
dyes such as the latter one do not completely stain the exoskeletons. As a result the 
external shape of the specimens is badly resolved, especially when the exoskeletons 








4. 5 Application of Fluorescence Dyes | 83 

are relatively thick, and the fluorescence of many of the stained internal structures 
(e.g., muscles) can be dominant in the resulting MIPs. To avoid this, the soft tissues 
can be digested using, for example, KOH, and the remaining exoskeleton can be 
precisely visualized with a non-specific fluorescence dye (e.g., Maruzzo, Minelli, 
and Fusco, 2009). A fluorescence dye, which was found to have a good potential for 
staining copepods, enabling taxonomic analyses of the exoskeleton using CLSM, is 
acidfuchsin (Kottmannet al., 2012). However, the staining properties involved have 
not been evaluated yet. Because acid fuchsin stains several different compounds 
and tissues, a respective study is necessary to understand in detail which parts of 
the copepods are stained. 

The exoskeleton material of arthropods contains, with only a few exceptions, high 
proportions of chitin. Therefore, when analyzing the morphology of crustacean 
exoskeletons it is recommended to use a fluorescence dye, which specifically labels 
chitin. For this, the chitin-binding probe (available from New England Biolabs, 
Ipswich, U S A) , a recombinant fusion protein that binds specifically to chitin, can be 
used. However, little experience in staining arthropod exoskeletons with this probe 
exists. Furthermore, the production and application of the chitin-binding probe are 
relatively time-consuming. For these reasons, it is generally preferable to use the 
fluorescent dye Congo red, which has been shown to successfully stain chitin in the 
exoskeleton of crustaceans and in the cuticle of polychaetes (Figures 4.7 and 4.8a; 




Figure 4.7 CLSM MIPs showing marine 
copepods and a marine polychaete stained 
with Congo red. (a) Ventral view of the 
calanoid copepod Femora tongicornis. The 
Congo red fluorescence is shown in yellow, 
and the autofluorescence of resilin is shown 
in blue, (b) Lateral view of a harpacticoid 



copepod of the genus Mesocietodes. (c) Ven- 
tral view of a polychaete of the genus Ari- 
cidea. In (b) and (c) the fluorescence of 
Congo red is shown in the glow mode of the 
software Leica LAS AF. Scale bars = (a) 200, 
(b) 100, and (c) 250 pm. 
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Figure 4.8 CLSM MIPs illustrating the stain- 
ing of different components of the marine 
calanoid copepod Centropages hamatus. (a) 
Ventral view of the anterior part of a spec- 
imen stained with Congo red. The fluores- 
cence of the dye is shown in gray, and the 
autofluorescence of resilin is shown in blue. 



(b) Ventral view of the anterior part of the 
muscle system of a specimen stained with 
a conjugate of phalloidin and the fluores- 
cence dye tetramethylrhodamine-5-(and-6)- 
isothiocyanate. The fluorescence of the dye is 
shown in green. Scale bars = 75 pm. 



Michels and Biintzow, 2010). Congo red is very easily and rapidly applied (Michels 
and Biintzow, 2010), and the fluorescence of the stained structures is rather stable 
and allows for long scanning times. These properties make Congo red a very suitable 
dye for detailed CLSM analyses of the exoskeleton morphology within taxonomic 
studies. Furthermore, the fluorescence of Congo red can efficiently be visualized 
in combination with that of other dyes and with autofluorescences to obtain 
information about the material composition of arthropod exoskeleton structures 
(Figures 4.7a and 4.8a; Michels and Gorb, 2012; Michels, Vogt, and Gorb, 2012). 
Bacterial chitinases labeled with fluorescence dyes were shown to efficiently stain 
chitin in aquatic fungi due to a chitin-binding domain (Wurzbacher and Grossart, 
2012). In this study chitin present in other organisms, including zooplankton, 
was also stained. However, the staining of crustacean exoskeletons has not been 
assessed in detail so far. 

In the following, a few additional examples are described of applications of 
fluorescence dyes for CLSM imaging of the morphology of marine organisms. 
However, owing to space constraints only a small overview is given. One common 
and important application is the staining of F-actin using phalloidin conjugates. 
Phalloidin is a toxic peptide isolated from the mushroom Amanita phalloides. It 
selectively binds to F-actin in fixed cells, and by using phalloidin conjugated to dyes 
cell cytoskeletons can be fluorescently labeled. In addition, such conjugates are 
often applied to visualize muscle systems of organisms (Figure 4.8b; e.g., Muller 
and Worsaae, 2006; Bergter, Brubacher, and Paululat, 2008; Winchell, Valencia, 
and Jacobs, 2010; Worsaae and Rouse, 2010). Nervous systems of organisms can 
be visualized using, for example, monoclonal anti-acetylated a-tubulin as antibody, 
which can be labeled with a second fluorescing antibody (e.g., Winchell, Valen- 
cia, and Jacobs, 2010; Worsaae and Rouse, 2010). 4',6-diamidino-2-phenylindole 
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(DAPI) is commonly used to stain DNA and visualize bacteria cells (e.g., Davidson 
et al, 2004; Garneau et al., 2009; Romanova and Sazhin, 2011) and cell nuclei (e.g., 
Worsaae and Rouse, 2010). This enables the user to show the overall morphology of 
organisms. DAPI simply has to be put into the embedding medium together with 
the specimens, making it very easy to apply this dye. A disadvantage of DAPI is that 
its excitation maximum is also located in the UV range of the light spectrum so that 
not all CLSM users can apply it (see above). As alternatives other nucleic acid stain- 
ing dyes are available. Among them are TOTO®-3 iodide (e.g., Winchell, Valencia, 
and Jacobs, 2010) and SYTO® dyes. The latter are available in several versions 
with several different excitation and emission properties so that the selection of the 
dye can be adjusted to the imaging needs. SYTO® dyes are known to efficiently 
stain bacteria (e.g., Davidson et al., 2004; Romanova and Sazhin, 2011). Among the 
different SYTO® versions, SYTO® 9 is often used in combination with propidium 
iodide (e.g., Davidson et al, 2004; Romanova and Sazhin, 2011). While SYTO® 9 
penetrates and stains both live and dead bacteria cells with intact and damaged 
membranes, propidium iodide penetrates only dead bacteria cells with damaged 
membranes. This enables the user to distinguish between live and dead bacteria. 

Today, studies on the influence of ocean warming and acidification on marine 
organisms have become very important. In this context, the visualization of calcium 
carbonate structures is often of interest. The fluorescence dye calcein, which binds 
to calcium and is incorporated into newly forming calcium carbonate structures, 
can be used to visualize the morphology and development of such structures in 
living organisms (e.g., Moran, 2000; Tambutte et al, 2012). 



4.6 

Surface Topography Analyses 

In recent years CLSM has also become a frequently used technique in materials 
science. In this scientific discipline CLSM has the potential for several different 
applications (Hovis and Heuer, 2010), the most common being analyses of surface 
topographies. By contrast, in biological science disciplines such analyses are not yet 
prevalent although many potential applications are conceivable. In the following, the 
methodological prerequisites of CLSM topography analyses of biological surfaces 
are described. 

Fluorescence light can generally be used for surface topography analyses. How- 
ever, in many situations this is not suitable because either the fluorescence intensity 
is not high enough or the bleaching of the fluorescence is too intense to yield good 
results (or even both effects are present in combination). Furthermore, compared 
to deeper tissue layers, the outermost tissue layers of specimens often exhibit lower 
fluorescence intensities, making precise analyses of the surface topography very 
difficult or sometimes even impossible. As already mentioned, besides fluorescence 
light the laser light reflected from the specimens can be used for CLSM visualiza- 
tions. In general, all structures with good reflective properties can be successfully 
visualized in this way, and for surface topography analyses the use of reflected 
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laser light commonly is the method of choice. In transparent specimens, internal 
surfaces with suitable reflective properties can be analyzed, but most investigations 
typically focus on outer surfaces. 

The investigated surfaces must exhibit a pronounced reflectivity. Although 
surfaces of transparent samples can be successfully analyzed, transparency often 
causes problems. For example, when analyzing the surface topography of diatom 
frustules a considerable proportion of the laser light is transmitted into the 
transparent frustules. Owing to the frustules’ complex geometries, this light is 
reflected at the frustule boundaries in various different directions, often resulting 
in interferences, which affect the detected signal. In such a situation it is necessary to 
increase the reflectivity of the outer surfaces. In general, the larger the proportion 
of the light reflected at the surface and the smaller the proportion of the light 
transmitted into the specimen the higher are the efficiency and the quality of the 
surface topography analyses. The reflectivity can be enhanced by sputter-coating 
the specimens with metal layers. For example, the snail shell and the diatom 
frustules shown in Figure 4.9 were sputter-coated with a gold palladium (80/20) 
layer. While for the snail shell a 20 nm thick layer proved to be very suitable, the 
diatom frustules had to be coated with a 50 nm thick layer to yield good results. In 
this context it is very important to ensure that the thickness of the coating is not 
too large compared to the surface structures. Otherwise, the results of the surface 
topography analyses would be affected and would no longer be correct. 

When trying to improve the reflectivity of outer specimen surfaces the embedding 
medium is also important. Surface topography analyses of specimens embedded 
in media such as water, glycerin, and immersion oil are generally possible and 
can yield good results. Sometimes, however, the application of air embedding is 
necessary. If light hits an interface between two media with different refractive 
indices a certain proportion of this light will be reflected at the interface, and 
the rest of the light will be transmitted into the second medium. The larger the 
difference between the refractive indices of the two media involved the larger is 
the proportion of the light reflected at the interface (e.g., Pedrotti, Pedrotti, and 
Pedrotti, 2006). As the refractive index of air shows a greater difference from those 
of most biological materials than those of, for example, water or glycerin, it can 
be of advantage to dry the specimens and analyze them in air. For this, specific 
objectives designed for air immersion and preparations without a cover slip are 
available. However, this will of course only be reasonable if the specimens do not 
tend to exhibit drying artifacts. If the specimens possess low transparency and 
their outer surfaces are very reflective it will be advantageous to embed them in 
media with higher refractive indices. This allows for the application of objectives 
with higher numerical apertures and provides the opportunity to perform surface 
topography analyses with higher resolutions. 

In an optimal situation, the reflectivity properties of the specimen are so good 
that just one single surface, which is the surface of interest, reflects the light. The 
CLSM analysis system can then provide a resolution in the z-direction, which is 
much higher than according to the Rayleigh criterion in situations when two light 
points have to be separated from each other by the optical system (Hovis and 
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Figure 4.9 CLSM analyses of the sur- 
face topography of diatom frustules and 
a snail shell, (a-e, g) Frustules of marine 
diatoms of the genus Arachnoidiscus. (h, 
i) Protoconch of the shell of the marine 
snail Tornus subcarinatus. (a, b, g, h) Color- 
coded height maps, (c, d, i) MIPs, (e) scan- 
ning electron micrograph (Courtesy of Lars 
Friedrichs/Alfred-Wegener-lnstitut), and (f) 



height profile along the arrow shown in 
(g). The height maps illustrate the differ- 
ent heights of the surface structures with a 
color code ranging from blue via green, yel- 
low, and orange to red representing heights 
between Ojim (dark blue) and maximum val- 
ues (dark red) of 14.87|im (a), 15.81 [J.m (b) 
15.85 |im (g), and 215.23 p.m (h). Scale bars 
= (a-c, e, g) 40, (d) 20, and (h, i) lOOjim. 
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Heuer, 2010). To determine the exact z-position of the surface the system just has 
to detect the z-position with the highest signal intensity. The latter can be achieved 
with a precision similar to that of the microscope’s focus system. For example, in 
upright microscopes focusing is performed by moving the stage. Accordingly, the 
precision of measuring the z-dimensions of a specimen depends on the precision 
of the stage control system, which can be in the range of a few tens of nanometers. 
To allow for the application of this precision it is necessary to adapt the properties 
of both the optical sectioning and the z-stack collection of the CLSM system by 
making the optical sections thin enough and decreasing the distance between the 
centers of consecutive optical sections within the micrograph stack. For this, the 
use of a pinhole size of 0.3 AU and a z-distance, which is a quarter of the thickness 
of the respective optical sections, is recommended. As a result, the optical sections 
are very thin and the amounts of light detected from each optical section are 
rather small. Flowever, despite this, the signal qualities are still high because of the 
relatively high intensity of the laser light. 

Another aspect, which requires attention, is the adjustment of the detected signal. 
In the context of surface topography analyses it is absolutely necessary to avoid 
any oversaturation. This is because oversaturated signals would result in the loss 
of information about the respective surface structures. Accordingly, the surface 
areas concerned would not be visualized correctly, and the results would not be 
consistent with the real conditions. 

If all the aspects described above are considered and the imaging setup is well- 
adjusted the analyses will provide detailed information about the surface topography 
and enable precise measurements of the surface structures in 3D (Figure 4.9). 
Furthermore, based on the results, standard parameters (e.g., surface roughness 
parameters), which are well established to describe and compare the properties of 
surfaces, can be calculated. Besides color-coded height maps, which are typically 
used to display the results of surface topography analyses (Figure 4.9a,b,g,h), 
MIPS can be created from the results (Figure 4.9c,d,i). The structural information 
obtained from such MIPs is similar to that obtained from comparable scanning 
electron micrographs created at relatively low magnifications (Figure 4.9c-e). 



4.7 

Future Perspectives 

Although CLSM has, as demonstrated, great potential for efficient and precise 
morphological imaging of marine organisms and can be used in a large variety of 
scientific contexts, the number of projects with respective CLSM applications is 
still relatively small. Flowever, in recent years, knowledge of CLSM and its potential 
applications has increased and spread considerably in the scientific community, 
and many of the scientific institutions in the field of marine biology have become 
equipped with confocal laser scanning microscopes. Accordingly, today, many 
marine biologists have access to CLSM systems. In addition, the CLSM producing 
companies have greatly improved the operating software of the modern commercial 
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CLSM systems to make it more user-friendly and to enable inexperienced users 
to also successfully apply CLSM and yield good scientific results. Consequently, 
it is very likely that CLSM will be applied more often and gain more importance 
for morphological studies in marine biology in the near future. This chapter will 
hopefully contribute to such a development. 
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5 

Optical Projection Tomography 

Karl Caff, Luke McCormac Parker , Dee Lawlor, and Emmanuel C. Reynaud 



5.1 

Introduction 

In the Greek language, the word “tomos” can be translated as “section” or “slice.” 
The definition of the term “tomography” from Merriam-Webster’s Dictionary is: 

“a method of producing a three-dimensional image of the internal structures 
of a solid object (as the human body or the earth) by the observation and 
recording of the differences in the effects on the passage of waves of energy 
impinging on those structures.” 

Optical projection tomography (OPT) is an emerging technology that provides 
a novel approach to recreating three-dimensional (3D) images of small biological 
specimens. The technique was developed to fill an imaging gap that existed 
between [rMRI (p, magnetic resonance imaging) and confocal microscopy. Being 
most suited to specimens ranging in size from 1 to 10 mm across, the technique 
is similar to X-ray computed tomography (CT). In this chapter, OPT is explained, 
its most important applications in biomedical research explored, and comparisons 
drawn of its pros and cons with respect to various alternative imaging technologies. 



A Brief History of Optical Projection Tomography 

In 2001, Dr James Sharpe of the MRC Human Genetics Unit (HG U) in Edinburgh 
developed OPT as a new tool to generate a 3D atlas of the genes and proteins 
in mouse embryos. Since the confocal technique is typically restricted to samples 
thinner than 200 pm, there was no alternative technology to suit his requirements. 
His only option was the laborious and time-consuming technique of physically 
sectioning the sample into very fine slices; however, due to the scale of the sample 
this was, of course, also ruled out. The principle behind OPT technology is not a 
new idea as this approach had been used for the previous two decades in X-ray 
GT. The technology, however, had never been employed for optical imaging of 
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biological samples. Consequently, Sharpe began to build such a system using spare 
parts lying around the laboratory. He recorded multiple projections of the specimen 
through different orientations using a low numerical aperture {NA) (large depth 
of field) lens to gather as much information as possible in a single image. He 
concluded that his setup could reconstruct 3D images of whole specimens in size 
range 1-15 mm (Sharpe et al., 2002). 

The technique could also be combined with staining methods to label internal 
structures and, most importantly, the complex genetic activities in different tissues, 
simultaneously. OPT can image specimens that are too large for conventional 
confocal microscopy and too small for p,MRI. The OPT technique proves to be cost- 
effective because it is able to image biological structures in vivo without damaging 
the specimen in the process. 



5.2 

What Is Optical Projection Tomography? 

OPT can operate as eitheremission optical projection tomography (EOPT) or trans- 
mission optical projection tomography (TOPT). The two modes can be employed 
respectively to image fluorescent light emission from excited fluorophores or the 
magnitude of absorption of transmission light through biological samples (Wang 
and Wang, 2007). 

For EOPT, the specimen is illuminated with a specific frequency of light whose 
energy provides the threshold energy required to excite the fluorophores of interest 
in the sample. For example, green fluorescent protein (GFP) possesses a major 
excitation peak at 395 nm and a minor excitation peak at 475 nm. This means that 
GFP emits green light with frequencies of 600 THz, corresponding to energies 
of 3.9 eV, when excited by photons with energies of 5 and 4.18 eV respectively. 
Therefore, the excitation light source must generate light whose emission spectrum 
correlates with that of the excitation spectrum of GFP. The resulting detected 
signal intensity depends on the fluorescence emission intensity arising from the 
fluorophores, which itself depends on the intensity of the excitation light probing 
the fluorophores within the tissue structures (Lee et al., 2006). 

For TOPT, a broadband lamp situated behind a frosted screen provides a diffuse 
source of uniform illumination. Projected onto the specimen, this bright field 
illumination is detected on the opposite side of the specimen, where the intensity 
depends on the amount of absorbed light. In other words, the resultant image is 
essentially a projection that is dependent on the attenuation coefficients through 
the specimen: a “shadow.” Detected by a wide-field objective lens, and ultimately by 
a camera, the light rays propagate through the specimen in approximately straight 
lines called projections. Analysis software on a computer performs a calculation to 
measure the degree to which each of the individual rays incident on each pixel has 
undergone attenuation. This is carried out by measuring the number of photons 
detected along each projection and then reconstructing either the absorbance or 
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Figure 5.1 Principle of the application of 
light to a sample and its detection in optical 
projection tomography. The light from a light 
source is transmitted through a diffuser in 
order to apply a homogeneous illumination 
to the object of the complete field of view 



of the detector sided imaging system. A sin- 
gle projection contributes the data from the 
whole depth of the sample and, thereby, it 
already contains full volumetric information. 
(Reproduced with permission from Heiko 
Meyer). 



emission distribution by performing a back-projection algorithm called the inverse 
Radon transform function (Sharpe, 2004). 

The specimen to be visualized is typically embedded in a cylinder of agarose, or 
in a capillary, as in Figure 5.1, and is strategically positioned at the focal point of the 
objective lens along the optical axis. The longest axis of the sample is aligned with 
the axis of rotation and is perpendicular to the optical axis. This orientation provides 
the shortest optical path through the specimen and is therefore the optimal way of 
measuring the transmission through the specimen. The depth of field is optimized 
to cover the region from the center of the object to the edge of the object closest to 
the objective. The cylinder is precisely fixed in the center of a rotary stage. 

A series of 2D optical projections through the specimen are acquired at several 
different azimuths, from which the 3D structure of the sample can be compu- 
tationally reconstructed (Swoger and Sharpe). For each successive image that is 
recorded, TOPT requires that the projection data acquired from specific planes are 
collected hy a linear row of pixels and that the points located along a line parallel to 
the optical axis should be projected onto the same pixel of the CCD (charge-coupled 
device) array (Wang and Wang, 2007). 

To acquire 2D optical projection images, the sample, is rotated not through 180° 
as one might expect but through 360°. This is because images that are recorded 
180° apart are not mirror images since the depth of focus lies halfway from the 
center of the specimen to its front edge closest to the objective. This focal region 
was chosen as a compromise due to the limited depth of field held by all optical 
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imaging systems. However, this method yields excellent results, not only in terms 
of image quality but also in maximizing the amount of information obtained in as 
few images as possible, thereby keeping imaging time to a minimum (Quintana 
and Sharpe, 2011). 

Usually, 400 images are recorded throughout the 360° revolution (one image 
every 0.9°). It is apparent, however, that a higher angular resolution would result in 
a higher image resolution. As with most imaging techniques, a compromise must 
therefore be met between image resolution, acquisition speed, and processing, and 
for most practical purposes this is likely to be an angular resolution of approximately 
0.5° (Sharpe, 2004). 

An infinity-corrected objective lens allows the implementation of auxiliary com- 
ponents, such as a filter wheel or a variable aperture (iris) into the parallel optical 
path between the objective and the tube lens (Figure 5.2). The infinity-corrected 
objective lens shapes the diverging light rays into a collimated wave train. For 
EOPT, an emission filter ensures that only the light within a narrow frequency 
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Figure 5.2 Illustration of the setup for TOPT showing all the components of the OPT 
configuration. 
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Figure 5.3 Illustration of the setup for EOPT showing all the components of the OPT 
configuration. 



band corresponding to the fluorophore emission spectrum reaches the detector. 
The optical configuration for EOPT can be seen in Figure 5.3. An iris positioned 
between the filter wheel and the tube lens provides control of the effective NA or 
f-number of the optical system (Wang and Wang, 2007). The tube lens focuses the 
propagating electromagnetic field onto the photosensitive surface of the camera, 
which lies in the focal plane of the tube lens. 

The sensitive surface of the camera, typically a CCD array, consists of a large 
number of light sensing elements called pixels. These pixels operate as individual 
photodetectors and respond to incident light via the photoelectric effect. This 
process results in the liberation of electrons leading to the build up of a charge 
in the pixel element. In an ideal situation, the generation of photoelectrons is 
linearly proportional to the number of incident photons. Initially, each pixel in the 
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sensor array functions as a potential well, storing the electronic charge during the 
exposure to incident light. 

5.2.1 

Assembly of an OPT System 

An OPT consists of five basic units. The wide-field objective lens, together with 
the camera, makes up the detection unit and is essentially a wide-field microscope. 
There are two illuminahon units, each consisting of a light source. One source 
provides transmission illumination, while the other provides the energy to induce 
fiuorescence in the specimen. Both sources contain optical components for shaping 
the beam as necessary prior to illumination. The sample manipulation unit holds 
the sample and embedding media and moves it relative to the stationary optical 
setup. The fourth unit is the control unit that operates the hardware such as the 
motorized rotary stage, the motorized filter wheel, the camera, and controls the 
data acquisition process. The opto-mechanical parts are mounted on an optical 
table, and the whole system can be controlled by a program written in LabVIEW 
(National Instruments). 

5. 2. 1.1 Detection Unit 

The detection unit is similar to that of a conventional wide-field fiuorescent 
microscope. It consists of an objective lens, a filter wheel, an iris, a tube lens, and 
a camera. To increase the depth of field, the effective NA of the objective can be 
reduced by means of an iris positioned behind the objective (Sharpe et al., 2002). 

An infinity-corrected air objective lens is used (Greger et al, 2007), because the 
light rays exiting from the rear of the objective are parallel rather than converging, 
as in finite objectives. To form an image, the light rays need to converge. This is 
carried out with an additional lens called a tube lens. The tube lens focuses the light 
onto the image plane. 

5. 2. 1.2 Illumination Units 

The illumination unit for EOPT uses a beam of light from a lamp assembly. The 
collimated beam is expanded to the required diameter for the sample under study, 
by a zoom beam expander. 

The illumination unit for TOPT uses a white light source. A piece of frosted glass 
is inserted at the exit pupil of the lamp assembly so that the geometry of the wave 
front becomes diffuse, to provide uniform illumination to the sample. 

5. 2. 1.3 Sample Manipulation Unit 

Since the optical arrangement is fixed, the sample is centered with its longest axis 
perpendicular to its axis of rotation prior to scanning. A computer-controlled rotary 
stage rotates the sample in the plane transverse to the longitudinal axis of the 
sample. 
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5.2.2 

Illumination Sources 

OPT is often compared to CT as an imaging technology and the main difference 
between them is that OPT creates an image using visible instead of the X-ray 
wavelengths of light employed in CT. 

The most common source of illumination for OPT is the xenon/mercury bulb. 

The benefits of xenon light are that they give off a high level of luminance and the 
light produced is uniform over the lifetime of the bulb. This is a vital characteristic 
in imaging to ensure continuity. The light generated by xenon bulbs is similar to 
that of sunlight and can be used to excite a wide variety of fiuorophores. 

Laser light is the second most commonly used after xenon/mercury. The 
monochromatic light of a laser is beneficial for the excitation of specific fiuo- 
rophores. However, for some applications, a regular white incandescent bulb can 
be used. LEDs (light-emitting diodes) are becoming increasingly popular for light- 
ing and although many microscopy techniques use lasers for illumination, the 
LED has several advantages - firstly, they are inexpensive and have a low power 
consumption compared to conventional light sources such as xenon lamps, they 
have a long life time, and they are readily available and easily replaced (Albeanu 
et al., 2008). LEDs also come in a wide variety of colors, covering the visible 
spectrum of wavelengths. 

5.2.3 

System Capabilities and Limitations 

In previous years, the traditional method for imaging the inner structures of a 
tissue was to physically section the sample. By imaging successive layers of the 
sample, a 3D reconstruction could then be compiled. However, this technique is 
very laborious and time consuming. Novel technologies such as OPT remove the 
need for physical sectioning of a specimen by using the tomography technique 
to acquire optical projections from a range of different azimuths to create a 3D 
virtual reconstruction of the sample. The capabilities of OPT imaging have been 
invaluable to many fields of biology, particularly developmental biology, gene 
expression research, and investigating the inner structures of tissues and biological 
specimens. 

The capabilities of OPT for creating a 3D reconstruction are competitive with 
those of other technologies and it is a relatively easy and affordable technique 
to use. Imaging can be carried out at up to 10-15 frames per second (Wang 
and Wang, 2007), which enables the operator to explore the sample and gain 
good quality, detailed images without having to wait for long periods of time. A 
limitation of imaging in 3D is that some features can be obscured by others, but the 
real-time capabilities of the hardware and software can compensate for this. When 
the sample is imaged with the additional dimension of time, 4D information can 
be obtained, which makes this technology invaluable in studying embryology and 
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developmental biology (Boot et al., 2008). Quantitative information such as volume 
and length can also be obtained using software (Fauver et al., 2005). 

In addition to time-lapse 3D reconstructions, gene expression imaging illustrates 
an extra advantage of OPT. Prior to OPT the location of gene expression could 
not be investigated without dissecting or mechanically sectioning the sample. But, 
with the dawn of this novel technology, gene expression in whole embryos can 
now be imaged using fluorescent or traditional dyes, allowing localization of the 
distribution of gene activity through 4D imaging (Fisher et al., 2008). 

The limitations of OPT are mainly in regard to sample size and penetration. 
OPT has been optimized to image samples ranging from as small as 2.82 pm 
(Fauver et al., 2005) up to 15 mm; anything larger can be imaged with CT. Samples 
at the larger end of the scale may need to be chemically cleared with an organic 
solvent. This involves replacing the natural fluids contained within the cells of the 
tissues with a fluid that matches the optical refractive index with that of the cellular 
membranes (Fumene Feruglio et al., 2010). This significantly reduces the amount 
of scattering, aiding clear, crisp image quality. 

OPT can operate in either transmission or fluorescent emission mode, while 
thin samples may only require traditional staining (Sharpe, 2004). Samples can 
he stained with fluorescent dyes (for gene expression) or with commonly used 
stains such as methyl blue (for histological imaging). Sample preparation for 
OPT is relatively simple and not as time consuming as sample preparation for 
technologies such as SEM (scanning electron microscopy) or TEM (transmission 
electron microscopy). 



5.3 

Comparison with Other 3D Microscopy Techniques 

Single-plane illumination microscopy (SPIM), an implementation of light-sheet 
microscopy (LSEM, light-sheet fluorescence microscopy), is a technique with 
applications similar to OPT. SPIM, however, in contrast with OPT, projects a thin 
sheet of light in a direction perpendicular to the detection axis, whose illumination 
is confined to the region around the detection focal plane and thereby suppresses 
out-of-focus light (Ntziachristos, 2010). The illumination of only the focal plane 
reduces the amount of background because no out-of-focus regions of the sample 
are excited. 

The optical selection of “slices” removes the need for a reconstruction method, 
whereas OPT requires back-projection reconstruction to form a readable image 
(individual images must be correlated to remove scattering effects; they are not 
utilized separately as SPIM images can be). SPIM image quality can be improved 
by illuminating the sample from both sides. As with OPT, chemical clearing can 
he used with SPIM. SPIM, like OPT, can he used for extended imaging of samples 
in vivo under their natural environmental conditions and the specimen can be 
removed afterwards from the gelling agent unharmed. 
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SPIM, however, is limited to the detection of fluorescence and is not capable, 
as OPT is, of imaging intrinsic or extrinsic absorption (Rieckher et al., 2011). 
Inexpensive colored dyes can be employed with OPT for staining tissues or for 
imaging gene expression. 

Ultramicroscopy is the forerunner to SPIM - its modern version is almost 
the same as SPIM - and so most of the above comparison also applies between 
ultramicroscopy and OPT. The primary difference between ultramicroscopy and 
SPIM is the orientation of the system. In ultramicroscopy, the objective is situated 
above the specimen (Dodt, 2007), which is held horizontally along its longest axis. 

The light sheet remains perpendicular to the objective. While the positions are 
relatively the same, the orientation does have a key difference. As it is imaged from 
above, the sample can be subject to deformation, as the gel is not rigid enough to 
resist the effects of gravity. The gel can be deformed, sinking downwards, bending 
or flattening the gel and, in severe cases, deforming the image, which is further 
complicated as the downward bend remains as the sample is rotated. This can lead 
to inconsistencies that can hamper the image processing. In OPT the gel is held 
vertically, which minimizes deforming gravitational effects. 

5.3.1 

Confocal Microscopy 

Confocal microscopy is an implementation of epi-microscopy in that both the 
excitation and emission beam are directed through the same objective lens. This 
optical configuration results in an elongated point spread function (PSF - the 
response of an imaging system to a point source, causing a blurred image of 
unresolved objects) in the longitudinal axis and is said to be anisotropic. The 
confocal technique is presently the commonest form of microscopy for obtaining 
3D images. At the same time it is a rather inefficient way of imaging fluorophores, 
as a great amount of energy is expended while very little of the fluorescent light is 
acquired by the detector. Most confocal imaging techniques, such a laser scanning 
confocal microscopy (LSCM), employ sequential, point-by-point illumination of 
the sample with a laser (or several lasers), along the same axis as the detector, and 
reject the out-of-focus light by means of an iris or pinhole in front of the detector. 

This subjects the specimen to high levels of electromagnetic radiation along the 
path of the laser, which can lead to photodamage and fluorophore bleaching after 
a short imaging period (Kurtz, 2007). Added to this is the fact that not all of the 
excess light is excluded from the image, as the confocal technique cannot entirely 
remove out-of-focus light from outside the focal plane. This “flooding” of light 
does not assist in the imaging and is a limiting factor that must be taken into 
account for live cell imaging. The excess light also results in additional background 
noise, which reduces image quality. Complicating matters is the fact that reducing 
the pinhole diameter to improve the optical sectioning also decreases the intensity 
of light reaching the detector (Semwogerere and Weeks, 2005), requiring even 
more excitation light (with corresponding photodamage) to image the sample. In 
contrast, OPT illuminates the whole sample, but exposes the sample to much lower 
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intensity, thus resulting in lower phototoxic, photodamaging, and photobleaching 
effects on the sample. 

Confocal microscopy can offer a lateral resolution of about 250 nm, compared 
with that of about 800 nm for the axial resolution (Gustafsson and Agard, 1999). 
This is due to the oval shaped PSF produced by the optical system. With such a 
PSF, the depth of field is a thin sliver of distance along the optical axis (z-axis). 
Upon scanning a plane of the sample with the excitation energy, the entire 3D 
sample is bathed in the excitation radiation, exposing every fluorophore to this 
energy. This not only induces toxicity and photobleaching in the sample but also 
limits the resolving power of the instrument. OPT utilizes software to combine 
the data from images taken from all around the sample to ascertain the location 
of various elements. Thus, the lack of axial resolution of one view is supported by 
the lateral resolution of other projections taken from different angles, which are 
combined to form the 3D model. 

5.3.2 

Two- Photon Microscopy 

Two-photon microscopy utilizes the near-simultaneous absorption of two long- 
wavelength (low energy-high penetration) photons to excite the fluorophores at 
a particular point in a sample. Only where the two photons superimpose is there 
enough energy to generate fluorescence, as the two photons together can provide 
enough energy to excite a molecule that either alone could not excite. This provides 
very fine control over which point of a specimen is excited for imaging. The 
long-wavelength photons used can also penetrate further into a sample than is 
possible with most other optical imaging techniques, allowing larger specimens to 
be used, because the long wavelength light is scattered less by the constituents of 
the sample. Compared to single-photon confocal imaging, two-photon microscopy 
has an inherently high signal-to-noise ratio (Potter, 2005), as no light-rejecting 
pinhole is required to achieve optical sectioning. Through a different mechanism, 
the same can be said of OPT: very little out-of-focus light is produced (in the 
case of OPT this is because of the large depth of field used rather than nonlinear 
absorption effects). 



5.4 

Sample Preparation 

As with all microscopy techniques, various protocols have been developed for OPT, 
to suit the preparation of different specimens and to image different investigative 
targets (neuronal tissue, expressed genes, etc.). Standard procedures for fixation 
and staining of a specimen are generally suitable and are applied where appropriate 
to the experiment. Some procedures are, however, characteristic of OPT in general. 

Samples are most often mounted in a transparent gel prior to imaging. This is 
done to allow for the gel to be manipulated and imaged at any point in rotation 
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without the apparatus used to suspend the sample interfering with the imaging 
process (such as a clamp that may block the view at some angles or a slide that 
will interact with the light differently at each angle). It also provides a more natural 
three-dimensional suspension, as opposed to laying the specimen out flat on a 
surface (such as a slide or plate), so there is less effect on the morphology of 
the specimen (the gel supports the sample, gravity is not pulling it down onto a 
surface). The gel also interferes much less with the light than most flat glass or 
plastic surfaces. The standard gel used is 1% low melting point agarose (Sharpe 
et al., 2002). Agarose provides sufflcient support, has sufficient optical qualities 
(dependent upon purity), and has little effect upon the biological specimen. Low 
melting point agarose is specifically used as it has less risk of heat shock - the 
sample must be placed in the gel while it is still liquid, so a higher melting point 
equates to a greater risk of damage to the sample. The sample is placed in the 
liquid gel and carefully maintained (e.g., using forceps) at a position close to the 
center of the gel as it sets. 

An alternative method involves forming the gel within a syringe. To do this, the 
head of the syringe must first be removed. A suitable amount of molten gel is then 
drawn into the syringe, the syringe is then pressed onto the sample, and then more 
gel is drawn up (if the sample consists of individual cells or is sufficiently small it 
can be mixed into the molten gel and everything can be drawn into the syringe at 
once) . This places the sample in the middle of the gel, which is then allowed to set in 
the syringe. When the gel is set, it is gently pushed out of the syringe to form a ready- 
made gel cylinder. This method, however, depends on the ability to acquire a syringe 
of suitable size for the specimen (particularly in terms of the width - see below). 

The sample should not be touching the sides or bottom of the container and 
should be adequately covered by gel on all sides. When the gel sets, excess gel is 
cut away. The gel is cut to form a roughly cylindrical shape so that the longest 
axis of the gel is the same as the longest axis of the specimen. The gel should be 
approximately three times greater in diameter than the axial depth of the sample. 

For specimens that are not naturally transparent, a clearing agent must be used 
to allow the light to penetrate for imaging. The sample must first be dehydrated, 
such as with a gradient of alcohol. The most common clearing agent used is BABB 
(benzyl alcohol to benzyl benzoate), also known as Murray’s Clear (Sharpe et al.,). 
BABB is a 1 : 2 ratio mixture of benzyl alcohol to benzyl benzoate. BABB is toxic and 
so cannot be used to clear organisms or tissues for in vivo imaging. Some samples, 
such as Drosophila, may require bleaching (such as with hydrogen peroxide) prior 
to clearing to properly remove pigmentation (McGurk, 2007). 

A magnetic holder is then attached to the top of the gel (along the shorter axes). 

This allows the gel to be held in the chamber for imaging by a magnetic stage that 
can rotate the sample without interfering with the light. Holding the gel in this 
orientation also helps limit deformation of the gel due to gravity. The OPT chamber 
is filled with an index matching liquid (a liquid with a refractive index similar to 
the gel) to minimize the scattering of light from refraction or diffraction, creating 
a more accurate image at the detector. BABB is often used as an index matching 
liquid (Sharpe et al.,). 
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If the sample is going to be imaged in vivo then conditions suitable for the 
specimen must be maintained (e.g., the gel must be submerged in water for 
an aquatic specimen) both during and between imaging sessions. Anesthetic is 
generally added to the gel to minimize stress on the specimen. If the sample is 
aquatic, some agarose can be added to the water to increase viscosity, thereby 
minimizing movements of the specimen that may disrupt the imaging process 
(McGinty, 2011). 

5.5 

Image Processing and Analysis 

Digital image processing software offers an enormous collection of computational 
algorithms, which contribute in many different ways to the processing and analysis 
of digital images. The role played by software, in the framework of digital imaging, 
is a crucial aspect in the pipeline of digital image processing and analysis. Such 
is the case with the visual programming environment LabVIEW, developed by 
National Instruments, which has played, and will continue to do so, a major role in 
data acquisition and 3D reconstruction. 

One method of image processing is taken from a PhD thesis entitled “In vivo 
optical projection tomography in biological model organisms” (McGinty, 2011) 
and is briefly described as follows. 

With a custom-made LabVIEW program developed, the raw projections are 
imported as a tif-stack. The first step of the reconstruction process is to set 
the upper and lower threshold limits of the dynamic range. Background data is 
subtracted from the image stack, to minimize artifacts such as noise and hot pixels. 
In addition to the image stack, the acquired data is assigned with a header file that 
includes all of the parameters used in the experiment such as acquisition time, gain 
setting, chip size, binning factor, excitation power, excitation wavelength, aperture 
size, sample information, employed excitation, and fluorescence filters. 

The subsequent stage in the reconstruction process is to define the region of 
interest, along with a defined center of rotation and the frequency of the Radon 
filter required for the back-projection algorithm (Meyer, 2010). Once filtered, the 
processed files are saved in 32-bit raw format files. Three-dimensional reconstruc- 
tion can be rendered and visualized using commercial off-the-shelf volumetric 
rendering software. 

5.6 

Marine Biology Applications 

OPT is used primarily to attain 3D models of mesoscopic specimens, such as 
embryos, with microscopic resolution (Figures 5.4 and 5.5) (Sharpe, 2004). This 
allows for much larger samples than most other techniques will allow to be imaged 
in great detail. In this way, OPT serves much of the role of X-ray GT or MRI for 
specimens that are too small for those modalities, and forms a bridge between 
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Figure 5.4 (a-c) Frontal, sagittal, and 

transversal view of the superimposed recon- 
struction of the exoskeleton (bluish-green) 
and dsRed expression form the 3XP3 pro- 
moter in the muscular system of a Parhyale 
hawaiensis] (d) 3D spatial view, and (e) 



same as (d) but with reduced opacity in the 
exoskeleton. The sample was kept in halo 
carbon oil to match the refractive index of 
the capillary. Reproduced with permission 
from Heiko Meyer. 




Figure 5.5 3D reconstruction of the exoskeleton of P. hawaiensis shown in Figure 5.4. Red 
lines mark the transverse optical sections depicted on the right. Data acquired by white 
light illumination. Reproduced with permission from Heiko Meyer. 
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Figure 5.6 OPT imaging of Sepia officinalis larval stages, (a, b) Max-value projec- 
tions through the entire reconstruction; (c, d) single slice taken from the mid plane. 

Green = autofluorescence; white = absorption. Sample courtesy of Laure Bonnaud; OPT 
scanning and reconstruction courtesy of Laura Quintana. 

macro- and micro-imaging. OPT can, therefore, be used to study complex tissue, 
such as brain tissue (Kerwin et al., 2004), in greater detail, as the tissue is not as 
affected by cutting artifacts and the cells are in situ relative to one another. OPT can 
also be used to study the development of tissues in some whole organisms, such as 
squid larvae (Figure 5.6), embryos (Boot etal., 2008), orplants (Rieckheret af, 2011). 

The variety of dyes and fluorophores that can be used with OPT has the 
advantage of not only allowing the specimen structure to be observed (such as 
with autofiuorescence) but also the expression of a gene (or multiple genes) 
of interest within the specimen (Sharpe et al., 2002). The ability of OPT to 
utilize simple colored dyes allows for the application of assays on the sample for 
which no corresponding fluorescent dyes have been found, such as the whole- 
mount detection of the expression of mRNAs (Sharpe, 2003). It also presents the 
opportunity to image both fluorescence and absorbance in the same specimen 
(with the same instrument) in three dimensions. 

The ability to image some samples in vivo even allows for 4D imaging (Boot, 
2009). These methods can be combined to allow for the expression of genes in a 
living organism to be observed in 3D models over time (Riecker et al., 2011). 

OPT can be used for the phenotyping of mutants (Albeanu et al., 2008), as many 
mutations have an unclear effect upon the phenotype of an organism when viewed 
in a two-dimensional manner. The capabilities of OPT allow for the differences 
caused by a mutation to be identified quickly and much more easily. 

More recently, OPT has been applied towards other avenues, such as the study of 
cancer tissue cultures, to allow culturing without subtype bias (beeper et al., 2011). 
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This may allow researchers to quantify drug sensitivity in such cultures instead of 
using a predictive method. 

Three-dimensional imaging techniques such as OPT have opened new doors 
into studying the growth and gene expression of plant tissues - providing both 
visual and quantitative information. Traditionally, plant tissues are sectioned and 
stained for imaging but this has limitations for investigation of living cell function 
and retaining cell and tissue structures. An advantage of OPT is that it can detect 
non-fluorescent signaling, gene expression, and obtain histological information. 

Algae are single-celled or multicellular plants that can be found in most fresh 
or salt water habitats. Optical sectioning can be performed on samples up to 
15 mm thick (Truernit et al, 2008). Fluorescent imaging can be conducted by trans- 
fecting with fluorescent stains or by natural autofluorescence from endogenous 
fiuorophores. 

Zebrafish are a model organism of vertebrate biology, and are widely studied 
(Bassi et al., 2011), although due to their size (in the centimeter range) there 
are relatively few imaging modalities suited to studying them or organisms of 
a similar scale. Juvenile zebrafish are naturally transparent, thus providing the 
perfect opportunity to develop three-dimensional models of vertebrate biology in 
vivo and with high resolution through OPT. A specific development in OPT, called 
Jlow optical projection tomography (Bassi et al, 2011), was designed to take advantage 
of the opportunities presented by organisms such as zebrafish. The purpose of 
flow-OPT is to image the vasculature of the organism by detecting the movement 
of the blood cells (the use of anesthetic removes most other sources of motion from 
the specimen). An algorithm for motion analysis is applied to the images, allowing 
a map of the vascular system to be built up. 

Casper zebrafish that were 30 days post-fertilization and transgenic for 
tg(fiil:GFP)yl were originally used as they do not have skin pigmentation and 
only weakly scatter visible light (Bassi et al., 2011). Although flow-OPT does not 
require the use of a label, fluorescent tags can be used with previously untested 
organisms to verify results gained and to detect artifacts (such as absorbance 
from the skeleton or intestines) (Bassi et al., 2011), although many artifacts are 
compensated for by images taken from different angles. The flow-OPT system 
consists of a light source positioned behind a diffuser that in turn is adjacent to an 
iris. A telocentric illumination lens collimates the light beam prior to illumination 
of the sample. A telocentric objective lens collects the transmission light and 
focuses it on the photosensitive surface of a camera. To ensure the samples remain 
stationary, they are mounted in a tube, 2 mm in diameter, made from fluorinated 
ethylene propylene {FEP) filled with agarose, permitting rotation of the sample. 

FEP has the same refractive index as the surrounding water in the chamber and 
thereby minimizes optical aberrations and artifacts. 

In a given stack of successive images, the flow contrast is calculated at each 
pixel by means of a mathematical equation (Bassi et al., 2011), providing a data set 
that can be rendered into a 3D reconstruction of the specimen’s vasculature. The 
specimen’s vasculature can then be sectioned virtually along any plane and viewed 
in two, three, or four dimensions. 
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Flow-OPT offers the capability to image living organisms from the millimeter 
scale of drosophila embryos, for example, up to the centimeter scale of zebrafish. 
The technology is capable of imaging large volumes in a non-invasive manner for 
long periods of time, without the need for fluorophores. With acquisition times 
of approximately 3 min, the technology is a relatively low cost method that can be 
combined with laser speckle imaging (Briers, 2006), which it is hoped will permit 
imaging of larger and highly scattering specimens. 
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Electron Microscopy Techniques for Imaging Marine 
Phytoplankton 

Custaaf Hallegraeff 



Abstract 

It is only since the 1950s, when the electron microscope was commercialized, 
that we have been able to fully appreciate the ultrastructural detail of marine 
phytoplankton. While there is no single, generic protocol for preparing 
phytoplankton for electron microscopy, this chapter provides a guide to the 
general approaches used for high-resolution imaging of species such as 
coccolithophorids with their calcareous scales, diatoms with silica frustules, 
and dinoflagellates with and without cellulose armor. Advances in sample 
collection, algal culturing, and molecular genetics have paved the way toward 
redefining many new taxa of prokaryotic and eukaryotic micro-, nano-, and 
picoplankton. 



6.1 

Introduction 

In 1673 the Dutch pioneer microscopist Anton Van Leeuwenhoek, using crude glass 
lenses, was the first human to see minute creatures, which he called “animalcules,” 
in pond water. From the 1890s to the 1930s, expeditions by vessels such as HMS 
Beagle, Challenger, and Meteor charted this newly recognized microscopic plant 
and animal world, describing thousands of new plankton species from the world’s 
oceans. Both the methods of collection (improved plankton nets, sampling bottles, 
fixatives , culturing skills) and techniques of observation have dramatically improved 
in the past century. The best optical microscopes, limited by the wavelengths of 
visible light, magnify no more than 1000 x and cannot discriminate between objects 
closer than about 0.2 pm. Different forms of illumination such as dark-field, bright- 
field, phase-contrast, polarized, interference, or fluorescence microscopy can be 
used to improve the resolution of particular structural details. It has been only since 
the 1950s when scientists developed the much more powerful electron microscope 
that we have been able to fully appreciate the unique shapes and structures 
of these microorganisms. Ernest Ruska (awarded the Nobel Prize in Physics 
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1986) first developed the application of a beam of electrons in a microscope. 
With a wavelength about 100000 times less than the average wavelength of 
visible light, this offers an increase in resolving power to about 0.001 jim. In 
the scanning electron microscope (SEM), a beam of electrons scans the outer 
surface of objects previously coated with a thin layer of gold or platinum to make 
them conductive. This instrument can produce a wide range of magnifications, 
from 20 to 30 000 x, and creates a great depth of focus or three-dimensionality 
400-500-times that achievable by light microscopy. The first micrographs showing 
this striking three-dimensional imaging capability were obtained in Cambridge 
in 1952 with a resolution of only 50 nm. The next important development was 
to improve the secondary electron detector by adding a scintillator to convert 
electrons into photons, and pave the way for enhancing the signal-to-noise ratio. 
Nowadays, the field emission scanning electron microscope (FESEM) produces 
much clearer, less electrostatically distorted images with spatial resolution down 
to 1.5 nm, that is, three to six times better than a conventional SEM. In another 
type of instrument, the transmission electron microscope (TEM), electrons pass 
through thin biological specimens, such as thinly silicified diatom cell walls or 
thin slices of cells embedded in epoxy resin and sectioned by means of a diamond 
knife in an ultramicrotome. In this way, magnifications up to lOOOOOx can be 
achieved. Starting in the 1960s Manton and Parke (1960) and coworkers (Leedale, 
Leadbeater, Dodge, and Moestrup) made important contributions to elucidate the 
fine structure of microalgal groups (Dodge, 1973), but TEM skills have become 
increasingly rare today. TEM sections are critical, however, in defining new taxa 
of prokaryotic and eukaryotic picoplankton (e.g., Synechococcus, Prochlorococcus, 
Ostreococcus; Chretiennot-Dinet et al., 1995). 



6.2 

Collecting and Processing Specimens 

Qualitative sampling in oceanic plankton surveys can be achieved using 37 pm 
plankton nets, either towed horizontally or used in a free-fall mode (Heron, 
1982). Eor inshore collections a small conical net (23 cm mouth diameter, 40 cm 
length, 4.5 cm cod end diameter) of 20 pm monofilament nylon is often used. 
Another sampling device is the continuous plankton recorder (CPR). This collects 
plankton continuously by seawater entering the instrument through a square 
aperture onto a 270 pm filtering silk mesh, which is then wound onto a storage 
spool in a tank containing 4% formaldehyde (Richardson et al, 2006). In contrast, 
quantitative sampling, which is needed, for example, when assessing changes in 
the abundance of potentially harmful phytoplankton, requires the collection of 
“water bottle’’ samples from various depths using 1, 5, or 81 Niskin bottles. Water 
samples of at least 1 1 are preserved and concentrated by settling or centrifugation. 
Eor some purposes a continuous plankton centrifuge (15 000g; flow rate 100 ml 
min“^) can be used to concentrate 21 to about 10ml (Davis, 1957). To obtain 
sufficient material of some rare species, the establishment of enrichment cultures 
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is invaluable. One milliliter of concentrated sample, obtained after continuous 
centrifuging, is inoculated into enrichment media, including dilutions of modified 
media (Andersen, 2005). 



6.3 

Light Microscopy 

It is always best to examine living phytoplankton microscopically as soon as possible 
after collection, either using bright-field or differential interference contrast, paying 
attention to swimming behavior and chloroplast colors. Alternatively, samples can 
be preserved for later examination, a commonly used preservative being Lugol’s 
iodine fixative (potassium iodide plus iodine; Sournia, 1978). However, this can 
overstain many algal cells, thus obscuring details that are important for their 
identification. For some species 1-2% formaldehyde (buffered with hexamine) 
or, for more delicate taxa, 2% glutaraldehyde (buffered with phosphate buffer 
and stored in a refrigerator at 4°C) are preferred. Both fixatives should only 
be used in well-ventilated rooms. Diatom net samples can be preserved in 2% 
formalin neutralized with hexamine, then rinsed with distilled water to remove 
salt and preservatives, and cleaned of organic material by overnight oxidation with 
potassium peroxodisulfate at 60 °C (Ma and Jeffrey, 1978). For mucilaginous or 
oil-rich diatoms more rigorous oxidants such as hydrogen peroxide and potassium 
permanganate, or potassium permanganate and hydrochloric acid, may be needed 
to remove organic contents. Cleaned diatom cells can be mounted in coumarone or 
Norland optical adhesive (refraction index 1.6-1. 7) and inspected by phase-contrast 
light microscopy. For armored dinofiagellates the use of the fluorescent cellulose 
stain Calcofiuor White M2R has become a valuable aid in the elucidation of plate 
tabulation (Fritz and Triemer, 1985). A major recent advance is the widespread 
availability of high-quality digital cameras (e.g., Zeiss AxioCam HR). This allows 
for on the spot verification of the successful capture of images, which then can be 
digitally shared with international colleagues for taxonomic verification. 



6.4 

Sediment Cyst Surveys 

For species such as dinofiagellates that produce resting spores (cysts; Figure 6.1), 
the examination of sediment samples is also recommended. The choice of sampling 
site should be based on an inspection of local bathymetric maps (depth contours) 
and knowledge of local currents. Black undisturbed sediments from deep basins 
offer better opportunities for cyst surveys than coarse sandy sediments characteristic 
of strong current regimes. Perspex coring tubes (20-40 cm long, 4.5 cm diameter) 
can be pushed into the sediment by SCUBA divers and then withdrawn and capped. 
Bottom samplers such as dredges or grab buckets often lose the lighter material at 
the sediment surface and are less suitable. A modified Craib corer (Craib, 1965) has 
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Figure 6.1 JEOL-35C SEM; air-dried; sediment sam- 
ple from Tasmanian waters; cyst of the dinoflagellate 
Conyaulax digitata, showing diagnostic germination open- 
ing (archeopyle); cell 45 |im long. 



also been used to collect undisturbed sediment cores. The uppermost 6- 10 cm of 
sediment was carefully extruded from the coring tube and mixed with filtered sea- 
water to obtain a watery slurry. Subsamples (5-10 ml) were sonicated for 2 min to 
dislodge detritus particles. The sample can then be screened through a 90 pm sieve 
and collected onto a 20 pm sieve, and the remaining fraction panned to remove 
denser sand grains and larger detritus particles. Sediment samples with compara- 
tively few cysts are best concentrated using sodium polytungstate density gradient 
centrifugation (Bolch, 1997). Individual cysts can be germinated in tissue wells with 
2 ml filtered seawater with nutrients added to confirm taxonomic identification. For 
some purposes destructive palynological sediment treatment techniques, involving 
the addition of 10% hydrochloric acid (to remove calcareous fossils) and heating 
with concentrated hydrofluoric acid, should be used (Sargeant, 1974). 



6.5 

Transmission Electron Microscopy 

Cleaned diatom samples prepared as above can be pipetted onto coated copper 
grids (400 mesh per in.), dried, and then immediately examined in TEM at 100 kV 
(Figures 6.2 and 6.3). For the study of small flagellates, 21 water samples should 




Figure 6.2 Philips-515 TEM; potassium peroxodisulfate cleaned; field sample from Gulf of 
Carpentaria, Australia; diatom Thalassiosira partheneia, with a single small marginal labiate 
process located in between two larger marginal strutted processes; 10pm diameter. 
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Figure 6.3 Philips-515 TEM; potassium peroxodisulfate cleaned; field sample from East 
Australian Current; detail of areola chamber of large diatom Coscinodiscus with internal sieve 
membrane (cribrum) and external foramen. 

be first screened through a 20|tm mesh to remove larger particles and then 
concentrated to 8-10 ml using filtration or a continuous plankton centrifuge. A 
drop of this concentrate can then be pipetted onto formvar- or parlodion-coated 
TEM grids. Samples can be fixed with 1 % osmium tetroxide vapor for 15-25 s 
and allowed to settle for 10-15 min before removing excess liquid using dental 
absorbent points. Caution: osmium tetroxide fumes are highly toxic and impact 
on the eyes in particular! Grids should be washed in distilled water and allowed to 
air dry. Grids can then be shadow cast (Moestrup and Thomsen, 1980) with Au:Pd 
(60 : 40) at an approximate angle of 30° in a Dynavac coating unit before viewing 
using the a TEM at 60 kV (Eigures 6.4 and 6.5). Until recently, the resolution of SEM 




Figure 6.4 Philips-515 TEM; Lugol fixed; air dried; field sample from Southern Australian 
coastal waters; haptophyte Pieurochrysis carterae with thinly calcified oval coccolith scales 
(cricoliths); cells 12-1 7 |tm diameter. 
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Figure 6.5 Philips-515 TEM; osmium tetroxide fixed; shadow cast using Pt; field sample 
from Tasmanian waters; body scales of haptophyte ChrysochromuHna polylepis including 
small oval (0. 6-0. 7 x 0.4-0. 5 pm) and larger elliptical scales (1. 0-1.1 xO. 4-0. 5 pm). Image 
courtesy of Jeannie-Marie LeRoi, University of Tasmania. 




Figure 6.6 Hitachi SU-70 FESEM FEI; buffered formalin preserved; field sample from North 
Atlantic using continuous plankton recorder; body scales of haptophyte ChrysochromuHna cf 
vexHHfera (0.9-1 .0 x 0.6-0.7 pm). 

did not approach that of the TEM, but this has changed with the advent of FESEM. 
Scale-bearing flagellates can now also be readily studied by FESEM (Figure 6.6). 



6.6 

Scanning Electron Microscopy 

Seawater samples (100 ml to 31) can be filtered gently onto Nuclepore™ filters 
(pore diameter 1 pm) immediately after collection. Other filter types produce less 
satisfactory backgrounds for SEM imaging. The filters should be rinsed with 100 ml 
distilled water to remove salt crystals and then air-dried. Small pieces of filter can 
be mounted on aluminium SEM stubs, coated with gold or gold-palladium (20 nm 
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layer thickness), and examined with a SEM at 20-25 kV. Instruments that we have 
worked with include JEOL JSM-35C, or JSM 840, Philips 515, FEI Quanta 600 
MLA environmental SEM, and Hitachi SU-70 FESEM (HallegraefF et al., 2010). 
Filtration immediately after collection, and the omission of dehydration and critical- 
point drying, minimizes the aggregation of detritus. For most species of armored 
dinoflagellates (Figures 6.7-6.10), diatoms (Figure 6.11), and coccolithophorids 
(Figure 6.12) structural deformation during air drying does not pose serious 




Figure 6.7 JEOL-35C SEM; air-dried; field sample from Papua New Guinea waters; recently 
divided pair of toxic dinoflagellate Pyrodinium bahamense, showing ornate cellulose plate 
tabulation; cells in 60 (im diameter. 




Figure 6.8 JEOL-35C SEM; osmium tetroxide fixed; critical-point dried; field sample from 
Adelaide coastal waters; toxic dinoflagellate Alexandrium minutum, showing diagnostic cellu- 
lose plate tabulation with triangular apical pore plate, connected to a polygonal first apical 
plate containing a ventral notch; cell 15 pm in diameter. 
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Figure 6.9 JEOL-35C SEM; air-dried; field sample from 
Papua New Guinea waters; diagnostic triangular apical 
pore of the toxic dinoflagellate Pyrodinium bahamense. 




Figure 6.10 JEOL-35C SEM; air-dried; field sample from 
Great Barrier Reef; diagnostic triangular apical pore of the 
toxic dinoflagellate Gambierdiscus toxicus. 




Figure 6.11 JEOL-35C SEM; air-dried; field sample from Sydney coastal waters; diatom 
Thatassiosira allenii; 15p.m in diameter. 
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Figure 6.12 Hitachi SU-70 FESEM; air-dried fieid sample from Southern Ocean; coc- 
coiithophorid Emiliania huxteyi covered by disc-iike coccoiiths (piacoliths) composed of 
T-shaped eiements radialiy arranged as in a spoked wheei; highiy calcified morphotype; cell 
diameter 5-lOpm. 

problems. For selected diatoms, cleaned net samples can be prepared for SEM. 
Delicate unarmored dinoflagellates can be fixed for 1 h at room temperature by the 
addition of equal volumes of 4% osmium tetroxide solution prepared in culture 
medium. The fixed culture then needs to be rinsed once in seawater, once in 
half-concentration seawater in deionized water, and once in deionized water, using 
centrifugation (600g) to separate the cells from the medium. The material should 
then be dehydrated using a methanol, ethanol, or acetone concentration gradient 
(e.g., 10, 30, 50, 70, 80, 90, 95, and 100% MeOH in distilled water, followed by 100% 
dry methanol and two changes of 100% dry acetone), soaking for 15 min at each 
concentration. Samples can then be critical-point-dried (Figures 6.13-6.15). In this 
process the solvent is replaced with liquid CO 2 and the temperature then raised to 
above the critical temperature (approximately 35 °C at a pressure of around 1200 psi) 
at which the liquid COj changes to vapor without a change of density and, therefore. 




Figure 6.13 EEI Quanta 600 SEM; osmium tetroxide fixed; critical-point dried; culture from 
Tasmanian waters; unarmored dinoflagellate Cymnodinium aureoium with horse-shoe shaped 
apical groove. Image courtesy Dr Chris Bolch, University of Tasmania. 
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Figure 6.14 FEI Quanta 600 SEM; osmium tetroxide fixed; critical-point dried; culture 
from Tasmanian waters; unarmored dinoflagellate Takayama tasmanica with sigmoid apical 
groove. Image courtesy Dr Miguel de Salas, University of Tasmania. 




Figure 6.15 JEOL-35C SEM; osmium tetroxide fixed; critical-point dried; cryptomonad 
Rhodomonas sp. with hexagonally arranged periplast plates and two emerging flagella. Image 
courtesy Tony Rees, CSIRO Marine &, Atmospheric Research. 



without surface tension effects that distort morphology and ultrastructure. After 
sputter coating, specimens for SEM can be stored under vacuum in a desiccator 
for long periods of time. As with light microscopy, the electronic capture of SEM 
images allows for on the spot verification of successful images, which then can 
be digitally shared with colleagues for taxonomic verification. As a general rule, 
electron microscopy is as much an art as a science, and phycologists must be 
prepared to experiment with new ways of handling cells if standard procedures 
appear inadequate. 
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Abstract 

In this chapter we describe computer-based tomographic methods using 
magnetic resonance imaging (MRI) and X-ray absorption as well as X- 
ray phase-contrast imaging techniques. The latter are based on refraction 
and near-held diffraction of X-rays and are gaining increased attention 
due to the increase in sensitivity they offer over conventional, attenuation- 
based X-ray techniques. The increased sensitivity is of particular interest in 
marine biology imaging, especially for soft tissue visualization and analysis. 
The basic principles of the imaging techniques are described, starting 
with an analysis of the image formation process, followed by a review of 
several tomography methods, such as MRI or X-ray imaging, and practical 
guidelines for parameter selection. Finally, two relevant examples of imaging 
are presented: imaging in marine biology (segmentation and visualization) 
and small animal (sea horse) imaging. These examples illustrate imaging 
under different conditions and using different reconstruction options. 



7.1 

Introduction 

Knowledge of marine life and its modihcations through historical time or as 
a consequence of ongoing climate change requires the accurate description of 
plankton and marine organisms from microorganisms to hshes. However, our 
biological knowledge in terms of, for example, physiology, biomechanics, evolution, 
ecology, and taxonomy of 70% of the Earth is extremely limited. The ultimate way 
to perform large-scale biological analysis is to achieve full three-dimensional (3D) 
imaging of every specimen collected. 

Recent developments have allowed tomographic methods to be used for scientific 
study in a wide range of fields. In this chapter we will focus on recent applications 
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of different types of tomography in marine biological imaging. We will review 
the fundamentals (essential base) of tomography and show the contrast formation 
models. We will illustrate the different experimental approaches used to study a 
marine organism and give some practical advice for image acquisition, parameter 
selection, and reconstruction. 

Computer-based tomography is emerging as a qualitative and/or quantitative, 
3D imaging technique used in a wide range of applications. The term “tomography” 
actually represents a large panel of methods. Magnetic resonance imaging (MRI) 
is devoted to the examination of soft tissues. On the other hand. X-ray computed 
tomography (CT) allows very high spatial resolution. X-Ray phase-contrast imaging 
(PCI) techniques offer a substantially higher sensitivity than standard, attenuation- 
based X-ray microtomography (|xCT). This offers new possibilities, especially in 
marine biological imaging, where |xCT often lacks the sensitivity needed to image 
soft tissues in the presence of strongly calcified tissues - PCI overcomes this issue. 
As there are several ways to obtain phase contrast with X-rays, we present the most 
important ones. We will review the first applications in marine biological imaging 
and discuss the prospects for further development and application of the technique. 



7.2 

Magnetic Resonance Imaging 

Microscopic magnetic resonance imaging (micro-MRI, MRM, or [rMRI) (Glover 
and Mansfield, 2002) is one of the best imaging methods available for assessing 
soft tissues structures and monitoring biological processes as they occur in pivo 
(Van der Linden et al., 2004). This technology does not rely upon ionizing radiation, 
yet it offers a spatial resolution of tens of microns (Glover and Mansfield, 2002; 
Tyszka, Fraser, and Jacob, 2005), thus exhibiting clear advantages over other 
imaging methodologies, such as positron emission tomography (PET). MRI is a 
non-invasive in vivo (Hart et al., 2003; Pohlmann et al., 2007), as well as post- 
mortem, imaging technique that allows the study of “soft” tissue (e.g., brain, 
bone marrow, muscle, etc.) (Tyszka, Fraser, and Jacob, 2005; Pohlmann et al., 
2007). It can be used to map anatomical structure (anatomical magnetic resonance 
imaging, aMRI) (Narasimhan and Jacobs, 2002), or as a monitor of organ function 
(functional magnetic resonance imaging,/MRf) (Lorenzetti et al., 2010). Moreover, 
magnetic resonance spectroscopy allows one to track various metabolic processes. 
The advent of high-magnetic-field MRI systems for small animals, that is, with 
an average magnet bore size that fits an adult rat body, has made this technique 
accessible for the study of smallest animal models (Hart et al., 2003; Tyszka, Fraser, 
and Jacob, 2005; Null et al., 2008). 

7 . 2.1 

Experimental Setup 

MRI is based upon the nuclear magnetic resonance principle. Certain nuclei, 
such as hydrogen nuclei (proton), for example, have a weak magnetic moment or 
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spin (Figure 7.1a). This magnetic moment allows the observation of a magnetic 
resonance signal caused by transitions between nuclear spin levels. Two hydrogen 
atoms are present in a water molecule, which is the principal constituent of living 
organisms. Their protons have the same resonance frequency and give one MR 
signal characteristic of the corresponding water molecule. Such MR signals are 
used to carry out anatomical and functional studies by MRI. 

The sample is placed in a static magnetic field, Bq (Figure 7.1b), which polarizes 
the populations of hydrogen nuclei (Glover and Mansfield, 2002) so that a bulk 
magnetization Mq appears, which is proportional to the number of hydrogen nuclei 
(N) and to Bq (Mq « Ny^ft^Bg/TfegT, where y is the gyromagnetic coefficient, ft 
is Planck’s constant divided by 27t, feg the Boltzmann’s constant, and T is the 
temperature). This magnetization, which is flipped using a radiofrequency pulse 
created by a radiofrequency probe (Figure 7.1c), returns along the direction of 
Bq, inducing in a detection probe a signal of which pulsation Wq is proportional 
to the magnetic field intensity, following the Larmor frequency law: Wq = yBg 
(Figure 7.1d,e). 

The time constant, which describes how the magnetization returns to its equi- 
librium value, is called the spin-lattice (or longitudinal) relaxation time, Tj. This 
process involves an exchange of energy between the spins and the environment. 
However, it must be pointed out that additional relaxation processes occur, which 
involve interactions between the spins without any change in the relative popu- 
lation of the energy levels. They induce dephasing of their magnetizations in the 
plane perpendicular to Bq. This phenomenon is described by the time constant Tj, 
which is called the spin-spin (or transverse) relaxation time. Notably, T 2 is <Tj in 
biological tissues. Moreover, the heterogeneity of Bq in the sample also contributes 
to the decay of the transverse magnetization. This contribution is made by the time 
constant T 2 ^, whose value is given by \ = I/T 2 + 1/T2^. As a consequence of 

these two phenomena the Tj, T 2 , and T^* of water protons in a biological tissue 
can be measured. Notably, these relaxation parameters depend on the tissue and 
on Bq, and are used to obtain contrast in the MR images. For example, anatomical 
images are obtained using Tj or T 2 weighted images, using MRI pulse sequences, 
which minimize Tj* effects and avoid susceptibility artifacts. The value is also 
very sensitive to susceptibility variations; T 2 * weighted experiments are used, for 
example, in fMRI, based on the blood oxygen (paramagnetic) level in vessels and 
its effects on surrounding tissues. 

As shown above, when a sample is placed in a homogeneous static field Bq, all the 
nuclei have the same resonance frequency, and only one signal is recorded without 
spatial information. To form an image of the sample, magnetic fields gradients 
have to be used. They establish a linear relation between the spatial positions of 
the nuclei and their frequency resonances, so the spatial information is contained 
in the MR signal (Figure 7.1f). 

The quality of an image is also related to the spatial resolution, the signal-to-noise 
ratio (SNR), and the contrast between the different tissues or organs. These three 
parameters are not completely independent. 
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The Spatial resolution achievable depends on the product of the strength of each 
gradient and its application duration. Consequently, there should be no limit in 
spatial resolution. 

Actually, the spatial resolution is limited by the SNR, and the diffusion of the 
water molecules; in fact, the spatial resolution achieved is about 10 pm. However, 
for in vivo experiments the total experimental time has to be taken in account, and 
the question is “How to increase the sensitivity and resolution power of MRl?” 
Diffusion is a physical parameter that depends on the biological tissue studied, 
and cannot be changed. Even if the SNR depends on the biological sample, it also 
depends on the hardware. In fact, the SNR depends on the initial magnetization 
(Mq) linked to Bq, on the coils, and on transmitter/receiver circuits. It also depends 
on the local Bq homogeneity, linked to gradient and shim coil systems. Most 
improvements made to increase the sensitivity and resolution power of MRl 
concern magnets, gradient coils systems, and radio-frequency probes used for 
emission and reception. 

The contrast between two different tissues can be characterized by the contrast- 
to-noise ratio [C=(Si — where Sj and are the mean of the 

signal and standard deviation of the noise i= 1,2, respectively]. It is linked to the 
difference between signal intensities (contrast) in the different tissues, which can 
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Figure 7.1 Summary illustrating the basic 
principles underlying nuclear magnetic res- 
onance. (a) A fish submitted to a natural 
magnetic field; hydrogen nuclei are randomly 
aligned, resulting in a null resultant magnetic 
field, (b) A fish submitted to a strong magnetic 
field, Bq. Each hydrogen nucleus (proton) gives 
rise to a magnetic moment (11 = y I where / 
is the spin moment of the proton and y is 
the gyromagnetic ratio characteristic of pro- 
ton), which processes around Bq at the Larmor 
frequency Vq = yBQlln. Considering Cartesian 
coordinates with thez-direction along Sq, /z has 
a stationary component along z and a compo- 
nent rotates at the Larmor frequency in the x-y 
plane. In the fish, all the magnetic moments of 
protons have the same Larmor frequency. The 
are spread over the x-y plane, and there is 
no net magnetization d. The /xz components 
can be parallel or antiparallel to Sq. The differ- 
ence in population in the two states follows the 
Boltzmann distribution with a surplus in the 
lower energy state (parallel), and gives rise to a 
macroscopic magnetization Mq parallel to Bq. 
(c) A fish submitted to Bq, and to an oscillating 
fields^ (at Larmor frequency) perpendicular to 
Bq. a pulse, which consists of turning on 
during a given time then turning it off, flips 



the macroscopic magnetization in the x-y plane 
(M), and a MR signal can be detected, (d) After 
turning off B^ the magnetization processes 
around Bq and returns to the equilibrium state, 
(e) Two time constants T-j and T2 charac- 
terize the exponential return to equilibrium 
of the longitudinal M^{t) and the transver- 
sale Mj^{t) components, respectively. 7 ^ is 
the longitudinal or spin-lattice relaxation time. 
The spin-lattice relaxation involves an energy 
exchange between the spin and the lattice, with 
these two systems being coupled via molecular 
motion. is the transverse or spin-spin relax- 
ation time. Spin-lattice relaxation contributes 
to T2, but additional processes occur, which 
dephase the spins without energy exchange 
with the lattice. The value of these two relax- 
ation times depends on the considered tissue. 
In a fish, 7 ^ and T2 depend on the differ- 
ent organs. This property is used to produce 
contrast between biological structures in MR 
images, (f) The application of a linear magnetic 
field gradient, along a defined direction, asso- 
ciated with a selective pulse allows us to excite 
selectively protons contained in a given virtual 
slice perpendicular to the gradient direction, 
and to obtain a MR image of this slice. 
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arise from the difference of concentration, relaxation times, or diffusion properties 
of water molecules. These different contrasts can be used to readily discriminate 
the different tissues in order to obtain anatomical or functional data. Noticeably, 
the contrast-to-noise ratio between two tissues also depends on their own SNRs. 
Increasing the resolution power decreases these SNRs, but it also decreases the 
partial volume effects occurring in the border pixels, and allows us to readily 
discriminate these tissues. Below, we will look at the influence of these different 
hardware systems on the sensitivity and resolution power of MRI. 

7.2.2 

Hardware Improvements 

As the signal intensity is proportional to Bq, increasing Bq leads to higher sensitivity 
of the measurement. Specialized high-field MRI systems (4.7 < Bq < 17.6 T) are well 
suited for high-resolution studies on small animals. Moreover, they are equipped 
with small bore gradient systems able to reach large amplitudes, which readily give 
fast access to high spatial frequency information. However, the signal intensity 
also depends on the radio-frequency probe (also known as the coil or resonator) 
used, which has to match the sample. A coil can receive or transmit or do both. 
The most common types of probes are surface coils (high sensitivity but produce 
a non-homogeneous magnetic held) and birdcage type coils (lower sensitivity but 
produce a homogeneous magnetic field). For experimental studies at high field, 
they can be combined to optimize homogeneity of the emission and sensitivity 
of the reception. Another way to optimize the MRI measurement is to develop 
“phase-array” coils, which are commonly used for clinical studies (Mispelter, Lupu, 
and Briguet, 2006). 

Even dedicated high-field systems, usually employed for experimental studies, 
can also be performed at low held using standard clinical whole-body MR scanners 
and may be of interest for some studies, particularly those using clinical contrast 
agents. The low SNR available at the typical field strength of 1.5 T may be overcome 
thanks to, for example, the use of specially designed radio-frequency probes in a 
superconducting material (Poirier-Quinot et al., 2008). 

7.2.3 
Contrast 

To assess the morphology of marine organisms. Optimized SNR and resolution 
are often not good enough to assess morphology. If signal differences do not exist 
between different structures inside the image, the image will be flat and featureless. 
Thus an adequate contrast, or signal-to-background ratio, is required to distinguish 
structures of interest. For a given tissue the signal intensity depends on the proton 
density (water content), the relaxation parameters (Tj, T 2 , or Tj*), and the diffusion 
properties of protons. These quantities vary from one tissue to another and are 
responsible for contrast between tissues. Different MR imaging protocols may be 
used to control the degree of contrast that can be achieved (referred to as the image 
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“weighting”). In 1980 Hansen et al. published the first anatomically interpretable 
MR images of a normal living rat body (Hansen et al., 1980) and thus laid the 
groundwork for the many MR applications in small animals that soon followed. 

7.2.4 

Applications 

7.2.4. 1 Anatomical MRI 

Owing to the variation of Tj, T 2 , or proton density between the tissues, MRI 
makes it possible to visualize most body organs (Figures 7.2 and 7.3). It allows 
us to generate cross-sectional images detailing brain structure (gray matter, white 
matter) (Narasimhan and Jacobs, 2002), or other aspects ofbody morphology (Kabli 
et al., 2006; Null et al., 2008; Ziegler et al., 2008). Such images are recorded with 
spin-echo type sequences, which are rather insensitive to Bq heterogeneities and 
susceptibility effects. Noticeably, at high field (Bg> 4.7 T), anatomical images are 
based on T 2 weighting techniques, whereas below this value anatomical images 
are obtained with Tj weighting protocols. The typical pixel size obtained is between 
100 X 100 and 10 X 10 pm^. For example, anatomical images have been obtained, 
in vivo, on a whole mouse brain with a voxel size of 78 x 78 x 500 pm^ at 7 T in 
15 min, and in vitro on a fixed specimen of about the same size with voxels of 
43 X 43 X 43 pm^ at 9.4 T in less than 7 h. 

7. 2.4. 2 Functional MRI 

Neural activity in the brain is accompanied by local changes in blood flow and thus 
oxygen levels. As oxygen is paramagnetic, the T 2 * of the tissues are locally affected. 

The difference in Tj* relative to surrounding tissues causes a contrast between 
the tissues, referred to as the blood oxygen level dependent (BOLD) response 
(Ogawa et al., 1990). fMRI is now recognized as the most prominent research 
tool in cognitive neuroscience (Poldrack, 2008; Dolan, 2008). It can also be used 
on animal models (Silva et al., 2011). The fMRI images are often obtained with 
gradient echo type sequences {T 2 * weighting) and using an echo planar imaging 
{EPI) acquisition scheme to obtain images with a very good temporal resolution. 

7. 2.4. 3 Diffusion Tensor Imaging or Diffusion MRI (DTI) 

Water diffusion in tissue is an intrinsic physical three-dimensional process. For 
most anatomical studies its effect has to be minimized. However, for specific 
purposes, water diffusion in biological tissues at a microstructural level can be 
used deliberately to generate contrast (Basser and Pierpaoli, 1996) by a method 
named diffusion tensor imaging (DTI). DTI is a powerful tool, providing access 
to the mean diffusivity, diffusion anisotropy, and fiber orientation in tissues (e.g., 
neuronal and muscular tissues). It offers a more direct and non-invasive method of 
measurement of the local orientations in tissues compared with other conventional 
3D imaging techniques (Beaulieu and Allen, 1994; De Groof et al., 2006; Aggarwal 
et al., 2009). The most advanced application of DTI is fiber tracking in the central 
nervous system (Le Bihan, 2003). 
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Figure 7.2 High-field MRI experimental 
setup, (a) MRI volumic probe enclosing 
the scrawled cowfish species Acanthostracion 
quadricomis. (b) Vertical high-field magnet 9.4 T 
with integrated gradient and shim systems, (c) 
Top, middle, and bottom virtual coronal slices 
of the body start at the middle trunk and fin- 
ish at the level of the head (you can see the 
brain). Volume rendering of a scrawled cowfish 
in (d) dorsal and (e) lateral views. Note that 
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the dorsal part of the cowfish has been virtu- 
ally removed. Visually classification of features 
of the organs (muscles, brain, live, eyes, etc.) 
allow a virtual dissection of the specimen, (f) 
Iso-surface three-quarter view of the head with 
cutting planes of the velvet belly lantern shark 
Etmopterus spinax, and virtual slice; (g) coro- 
nal, (h) frontal, and (i) sagittal planes. Scale 
bar= 5 mm. 



The DTI sequence is obtained by inserting diffusion gradients in a spin-echo 
sequence, to link the proton signal intensity of a water molecule to its diffusion 
properties, more particularly diffusion rate and direction. At least six different 
gradient directions (30 are usual in experimental studies) must be used during 
acquisitions to allow calculation of the diffusion tensor. The diffusion tensor is a 
symmetric 3x3 matrix, the eigenvalues of which are related to the diffusivities 
along three directions given by the eigenvectors. The first eigenvalue represents 
diffusion along the preferred or primary direction, while the second and third 
relate to perpendicular directions. Noticeably, the different gradient directions are 
obtained from combinations of gradients applied in the three spatial directions. 
To minimize the acquisition time and avoid movement artifacts rapid schemes for 
acquisition are used (EPI or spiral). 

7. 2.4.4 MEMRI or Manganese-Enhanced Magnetic Resonance Imaging 

Manganese(II) (Mn^+), a well-known Ca^+ analog, has been used in the field of 
light microscopy in conjunction with fluorescent dyes such as fura-2 to indirectly 
monitor the influx of Ca^+ ions, which is an indicator of neuronal spiking activity. 
In addition, Mn^+ is also paramagnetic, causing positive contrast enhancement 
in Tj-weighted MRI images in tissues where it accumulates. These properties 
have led to the development of manganese-enhanced magnetic resonance imaging 
{MEMRI) for cerebral (Massaad and Pautler, 2011) and cardiac (Edelman, 2004) 
studies. However, as Mn^+ is neurotoxic it can only be used in experimental studies 
with well-defined protocols (Canals et al., 2008). MEMRI is principally used for 
three types of brain studies (Silva et al., 2004). Eirst, as Mn^+ accumulates in cells 
in an activity-dependent manner, it can be used to monitor activity in the brain. 
MnClj is infused while the brain is stimulated. In this technique, referred to as 
activity-induced manganese-enhanced magnetic resonance imaging (AIM-MRI), 
MnClj is bath-applied while the brain is stimulated. Second, the use of Mn^+ 
as an in vivo, trans-synaptic, MRI-detectable neuronal tract tracer has also been 
demonstrated (Tucciarone et al., 2009). In this case MnCl 2 is locally injected to 
the target organ. The third use of MEMRI is to enhance the contrast in animal 
brain cytoarchitecture using systemic injections. Taken together, results in the 
literature indicate that MEMRI provides an efficient and powerful in vivo method 
for following brain development in physiological and physiopathological studies 
(Gillet et al., 2010). 
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Figure 7.3 Comparison between syn- 
chrotron phase-contrast X-ray microtomog- 
raphy and microscopy MRI of the Tha- 
lassinidea shrimp Upogebia pusilla. Views of 
the iso-surface of the shrimp: (a) dorsal, (b) 
lateral, (c) ventral, and (d) three-quarters, 

7.3 

X-Ray Microtomography 



with cutting planes and coronals slices. Vir- 
tual slice obtain by X-ray phase-contrast 
microtomography: (e) coronal and (f) sagit- 
tal plane. Virtual slice obtain by a high-field 
MRI (g) coronal and (h) sagittal plane. Scale 
bar= 1 mm. 



X-Rays were discovered as early as 1895, and their immediate first practical 
application was in biological and medical imaging. Computed tomography (CT) 
was developed in the late 1960s and extends X-ray radiography from two to 
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three dimensions. This is achieved by acquiring projections (i.e., radiographs) 
of an object along different directions and computing a cross-sectional slices 
(Cormack, 1963; Hounsfield, 1972, 1973) through the object or a complete 3D 
volume reconstruction (corresponding to virtual slice stacks). Since the 1980s, 
this technique has been primarily used for medical applications [i.e., large-scale 
biological samples with a resolution of about 1 mm (Kalender, 2006)]. However, 
with the arrival of synchrotron radiation (SR) and high-resolution CT equipment, 
tomography has been considerably improved and samples can now be imaged at 
submicrometric resolution (Baruchel et al., 2008), with effective pixel sizes down 
to about 50 nm (hanger et al., 2012). 

7.3.1 
Sources 

7. 3. 1.1 Laboratory-Based Setups 

In contrast to synchrotron sources, laboratory sources can be installed in relatively 
confined spaces with small high-voltage generators (typically 50-450 kV). The 
electrons are produced by the thermionic emission of a heated filament at the 
cathode side, and are then accelerated and focused in a vacuum environment 
before hitting the anode. The collision of electrons on the anode material is 
responsible for the X-ray photon generation. Part of the radiation is produced 
by the deceleration of the electrons (bremsstrahlung or braking radiation) while 
another part of the radiation is released when electrons from a higher electron shell 
of the anode material atoms fill up vacancies in the lower electron shells where 
electrons were knocked out during the collision (characteristic radiation). The 
X-ray photons of the bremsstrahlung spectrum have an energy ranging from a few 
kilo-electronvolts up to the energy corresponding to the acceleration voltage (this is 
why laboratory sources are polychromatic). In contrast, the energy of characteristic 
radiation photons corresponds to the difference in energy of the initial and final 
orbital jump of the electron that filled the vacancy. The combination of both 
radiation types is the spectrum of an X-ray tube and is thus dependent on the 
acceleration voltage (setting in kilovolts of a tube) and the anode material. Typically, 
the anode is made of a metallic material with a high melting point, such as tungsten, 
because most of the energy of the colliding electrons is transformed into heat and 
only 1% into X-ray photons. This spectrum can be altered by placing filters between 
the source and the detector. 

X-Ray CT with laboratory sources is also often called cone-beam tomography 
(Figure 7.4b,d) since the photons that are produced in a small location in the anode 
material, the focal spot, are dispersed in all directions. The photons that are directed 
toward the 2D imaging detector placed in front of the source make up the field 
of view. For every object placed inside the field of view, a transmission image, or 
radiograph, can be recorded. As the beam is conic, the radiograph of an object that 
is placed close to the source will be projected as a magnified transmission image 
on the detector, this magnification M equals the relative difference between the 
source-object distance (SOD) and the source-detector distance (SDD). For this 
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Figure 7.4 Principle of 2D and 3D CT. {a, b) 

Geometry of acquisition system in 2D CT: (a) 
translation-rotation acquisition (first genera- 
tion); the attenuation of a thin X-ray pencil 
beam is measured on a single detector. The 
X-ray source and detector are then sequentially 
translated and rotated, (b) Fan-beam acquisi- 
tion (third generation). The attenuation of a fan 
beam X-ray source is recorded on a linear detec- 
tor. (c) Optical paths of X-rays in 3D parallel 
beam synchrotron radiation microtomography 
(SR-|xCT). (d) Optical paths of X-rays in 3D 
cone-beam desktop microtomography (ptCT). 

In both cases, when the sample is sequentially 
rotated, a set of radiographs is acquired. They 
are processed through a tomographic recon- 
struction algorithm to obtain the 3D digital 
image of the sample, (e)-(k) Illustration of 


reason, the resolution {R) of a CT-scan is a combination of the pixel size {d) of the 
detector, the magnification (M) of the object, and the size of the X-ray focal spot (s): 

d ( 1 

R= —-\-sll 

M \ M 

In this way, using a small focal spot and large magnification, small objects can be 
scanned with good image resolution. With present laboratory sources a spot size 
of < 1 \xm can be achieved. 

7.3.1 .2 Synchrotron-Based Setups 

A synchrotron is a particle accelerator that can be used to generate electromagnetic 
radiation and, in particular. X-rays. Synchrotrons used for microtomographic appli- 
cations in biological imaging are distributed worldwide and include the following: 
Canadian Light Source (Canada), the Advanced Photon Source APS, the Advanced 
Light Source ALS, and the National Synchrotron Light Source NSLS (USA); 
Shanghai Synchrotron Radiation Facility (SSRF); Pohang Light Source PLS (Korea), 
SPring-8, and the KEK Photon Factory (Japan); European Synchrotron Radiation 
Facility ESRF and Synchrotron Soleil (France), Angstrdmquelle Karlsruhe ANKA, 
Helmholtz-Zentrum Berlin HZB/BESSY, and Deutsches Elektronen- Synchrotron 
DESY (Germany), Elettra Sincrotrone Trieste (Italy), the Swiss Light Source SLS 
(Switzerland), and Diamond Light Source (UK); and the Australian Synchrotron 
(Australia). Compared to conventional X-ray sources, synchrotrons have the advan- 
tages of a wide energy range (4-200 keV), high photon flux, and a monochromatic 
{AE/E^lO^^-10^^), parallel, and partially coherent beam (Figure 7.4a,c; Betz 
et al, 2007; Baruchel et al., 2008). In terms of performance for |xCT, the main 
advantages of using SR over X-rays from laboratory generators are, first, the higher 
spatial resolution that can be obtained and, second, the higher density resolution 
(or better contrast) (Betz etal, 2007). In terms of sample preparation. X-ray imaging 
techniques can be non-destructive (Boistel et al, 2009) and selective to a wide range 



data acquisition and tomographic reconstruc- 
tion in 2D parallel CT. (e) For a given angle 
of projection, one projection is measured on 
parallel X-ray paths. A complete data set (sino- 
gram) includes all projections when the angle 
varies between 0 and 1 80°. (f) Original synthetic 
256 X 256 “fish” image and (g) corresponding 
sinogram (or Radon transform) of the “fish” 
image, (h) Tomographic reconstruction of the 
“fish” image from the FBP algorithm: each 
projection is filtered and then back-projected, 
(i) Reconstruction without filtering the projec- 
tions: a blurred image is obtained, (j) FBP 
reconstruction after using a limited number of 
projections: streak artifacts appear, and (k) FBP 
reconstruction from projections in the 0-90° 
range: distortions appear due to missing infor- 
mation. 
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of properties (density, chemical elements, chemical bonds, strains), and provide 
3D morphometric information with resolution down to nanoscales. 

Synchrotron facilities are very large installations, with circumferences rang- 
ing from about 100m (ANKA) to about 1500m (SPring-8). A synchrotron is a 
closed-orbit particle accelerator that, when built for the generation of X-rays, uses 
charged particles, such as electrons, to produce very intense light beams. The 
operating principle of a SR facility is schematically represented in Figure 7.5. As 
in a conventional X-ray generator, the electrons are produced by the thermionic 
emission of an electron gun. However, they are accelerated through the sequence 
of a linear accelerator {LINAC; Figure 7.5a) and, usually, a first circular accelerator 
(booster ring; Figure 7.5a) where they reach their final energy level (respectively 





Figure 7.5 Schematic summary of a third 
generation synchrotron facility. Third gener- 
ation synchrotron (a): electrons are emitted 
by an electron gun and first accelerated in 
a linear accelerator (LINAC) before being 
transmitted to a circular accelerator (booster 
ring) where they are accelerated to reach 
an energy level of several billion electron- 
volts. These high-energy electrons are then 
injected into the large storage ring where 
they circulate in a vacuum environment at 
a constant energy level. Beamline are consti- 
tuted of a beam source (b) and three cabins: 



(c) optics, (d) experimental, and (e) control 
cabin. Radiographs exhibiting absorption and 
interference patterns attributable to Fresnel 
diffraction are recorded at two distances: (f) 
close to the sample in absorption and (g) 
far from the sample to obtain phase con- 
trast; (f) and (g) are, respectively, absorption 
and phase-contrast images of a virtual trans- 
verse slice of Osmerus eperianus at trunk 
level. Bone tissues only are visible on the 
absorption image, whereas bone and soft tis- 
sues can be visualized on the phase-contrast 
image. 




7.3 X-Ray Microtomography 



137 



8, 6, and 2.75 GeV at the SPring-8, ESRF, and Soleil facilities) before entering a 
final accelerator, the storage ring, in which they are kept at constant operating 
energy. Electrons in the booster and storage ring travel close to the speed of 
light (figure 7.5a). The storage ring is a polygon made of straight sections angled 
together with bending magnets (dipole magnets). SR is produced whenever the 
electrons in the storage ring deviate from a straight travel path. In practice, this 
is the case in the bending magnets, but also in special arrays of magnets called 
insertion devices (undulators and wiggler), which are placed in the straight sections 
of the ring. A bending magnet provides a uniform magnetic field, whereas an 
insertion device provides a spatially periodic magnetic field. As bending magnets, 
wigglers, and undulators are considerably different, it is important to know their 
characteristic radiation spectra and choose the one best suited for a given exper- 
iment. Indeed, the electrons follow a bending path with bending magnets and a 
wiggling path with inserted devices (which produces more intense and tunable 
light) (figure 7.5b). Bending magnets and wigglers produce a continuous spectrum 
of X-ray wavelengths, whereas undulators produce a line spectrum. In any case, the 
X-ray beam produced by a synchrotron is many orders of magnitude brighter than 
that produced by a laboratory X-ray machine. Synchrotron X-rays are exploited in 
experimental stations (or “beamline”) located outside the radiation shielding of the 
storage ring. Each beamline includes an optical cabin, an experimental cabin, and a 
control cabin (Figure 7.5c-e, respectively). The optical cabin contains diaphragms, 
slits, filters, and a shutter that allows for control of the bandwidth, photon flux, and 
beam dimensions of the X-rays. Synchrotron X-ray sources deliver X-rays with a 
large energy spectrum range (white beam). A wavelength adapted to the specimen 
under investigation is selected by a suitable combination of absorption filters and 
mirror reflections (“pink” beam) and/or by a monochromator (e.g., a set of Bragg 
crystals or a multilayer system). In the experimental cabin one or several detectors 
record the signal produced as a result of the interactions between the X-ray beam 
and the sample. The researchers conducting experiments at a beamline control the 
experimental equipment and the data collection process remotely from a control 
cabin located near the experiment. In several synchrotrons, long beamline are 
available that deliver a relatively large beam that is several centimeters wide. This 
is particularly useful for imaging techniques such as [jlCT. For example, the exper- 
imental station of beamline ID19 at the ESRF is located 150 m from the source 
point in the storage ring, and the beamline of the Biomedical Imaging Center 
(RI Laboratory) at the SPRing-8 synchrotron in Japan reaches a record length of 
1km. SR can be used for many imaging methods, including (i) absorption p.CT 
[using the intrinsic sample contrast (Figure 7.5f) or with a contrast agent]; (ii) 
X-ray PCI (Figure 7.5g), which can be classified into (1) free-space propagation 
methods, (2) crystal interferometer methods, (3) analyzer-based techniques, and 
(4) grating-based methods; and (iii) X-ray microspectroscopy, or chemical imaging. 
The latest developments can give access to these methods down to nanoscale 
resolution. 
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73.2 

Sample Stage 

To position the sample in the field of view and to adjust magnification, most 
CT-systems are equipped with several motorized stages. On the top of these 
positioning stages a rotational stage, upon which the sample is fixed, is mounted. 
By rotating the sample, radiographs in different angular orientations can be 
recorded. These radiographs are then further used to compute tomographic cross- 
sections of the objects. Precise and repeatable positioning are crucial to enable 
good reconstructions, especially at higher resolutions. 

7.3.3 

Detector 

Most X-ray detectors currently used are composed of a scintillator (Figure 7.4c,d) 
and a light sensitive camera. The scintillator is a thin layer of material that emits 
light when interacting with X-ray photons. In this way, a portion of the X-ray photons 
are transformed into visible light that can be detected and recorded using a light- 
sensitive sensor (very similar to that of a digital photography camera). Scintillators 
can be made out of different polycrystalline or monocrystalline materials [cesium 
iodide (Csl), gadolinium oxysulfide (Gadox), yttrium aluminum garnet (TAG), 
lutetium aluminum garnet (LAG)], which have different sensitivities for X-ray 
photons with different energies. 

The scintillator can either be placed in direct contact with the sensor (this is the 
case, for example, in the flat-panel detectors often used in laboratory-based [jlGT) 
or it can be coupled to the sensor via an optical system, for example, a fiber-optic 
system or a lens system. Lens systems consisting of visible-light microscope optics 
are used in high-resolution synchrotron-based microimaging, where the physical 
pixel size of the available digital sensors is too large to reach the desired spatial 
resolution with direct coupling, and magnification of the image created by the 
scintillator is therefore needed before it is recorded by the sensor. 

Another important aspect is the sharpness of the image obtained. The visible 
light produced by the scintillator will propagate and scatter inside the scintillator 
layer; and so will secondary particles generated upon impact of the X-ray photons 
in the scintillator. These effects lead to blurring of the image. The thicker the 
scintillator, is the higher the probability of internal scattering and, therefore, the 
stronger the blurring. Moreover, in a microscope configuration the scintillator 
thickness should not exceed the depth of focus of the objective lens. For these 
reasons, the best resolving scintillators are usually rather thin. 

The light-sensitive sensor (Figure 7.4c,d) itself is typically a solid-state device, such 
as a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor 
(CMOS) system, similar to the sensors used in digital photography. It is actually a 
2D array of very small sensor elements that translate the incident light photons into 
a digital signal (Figure 7.6c,f). Each of these small sensor elements will represent 
one picture element (pixel) in the resulting radiography image. These sensors 
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Phase contrast: large sample-datector distance 



Figure 7.6 The two types of contrast mech- 
anism exploited in X-ray imaging of the 
anatomy of a sample (a), that is, absorption 
contrast (b-d) versus phase-contrast (e-g). 
(b, e) X-ray beam; (c, f) signals detected by 
the detector; (d, g) radiography. Compari- 
son of the contrast obtained by absorption 



(d) and phase contrast (g): in absorption (d) 
only the mineral part of skeleton (white) and 
uniform contrast for soft tissues (gray) is 
obtained, while in phase contrast (g) fringes 
in white and black are seen at the interface 
of different soft and hard tissues. 



have an array size ranging from 500 x 500 pixels (0.25 megapixel) to 8000 x 8000 
pixels (64 megapixel). If a 1 cm x 1cm area of the scintillator is mapped on a 
sensor of 1000 x 1000 pixels matrix, each pixel will have a maximum size (d) of 
10 pm X 10 pm and the center-to-center distance between adjacent pixels (pixel 
pitch) is 10 pm. 

In synchrotron facilities, owing to the large source-object distance, no signifi- 
cant magnification can be realized with restricted object-detector distances and, 
therefore, the resolution of a CT scan will be determined by the pixel size and 
the blurring introduced by the scintillator. Consequently, very thin scintillators are 
used. When the thickness is reduced, the probability of interaction with an X-ray 
photon is also diminished, resulting in a lower signal-to-noise. But as the flux of 
a synchrotron beam is extremely high, this does not constitute a real concern, at 
least in applications where the specimens are not radiation sensitive. In practice, 
the spatial resolution of a high-resolution X-ray imaging detector at a synchrotron 
is limited to approximately 0.5 pm, both by scattering of secondary particles in the 
scintillator and by the diffraction limit of resolution in the visible-light microscopy 
optics that project the image created in the scintillator onto the surface of the digital 
sensor. Higher spatial resolution - going down to a few dozen nanometers - can 
be reached, but this requires the use of X-ray optical elements to magnify the image 
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even before it impinges on the scintillator. Such methods are usually referred to as 
“X-ray microscopy” methods. 

With laboratory sources, on the other hand, the flux is more limited and it is 
thus crucial to detect as many photons as possible by using a thicker scintillator. 
To obtain sharp images with such a thicker scintillator the pixels will have to be 
larger, but since the resolution is determined by the magniflcation, the pixel size is 
not important and so relatively large pixels are recommended. 

7.3.4 

Forward Problem (Contrast Formation) 

A radiograph of an object is an image containing information on the intensity of 
the X-ray beam after being transmitted through the object {!) that can be compared 
with the beam intensity without the object (7 q). Using digital detector systems, 
an image without the object in the held of view is recorded first (flat-field image) 
and each pixel of the X-ray detector will return a digital signal (counts) that is 
related to the number of X-ray photons that interact with the pixel. Since every 
pixel has a slightly different efficiency, and due to other fluctuating parameters, 
it is important to record such a flat-field, which represents the intensity ip of the 
beam and detector response in every pixel. Once an object is placed inside the field 
of view, the X-ray photons will have to travel through the object before hitting the 
detector. During travel through the object, an interaction between the photon and 
the object’s material may occur. If such an interaction does occur, the photon will 
not reach the detector, or reach it at a different position, and thus the intensity of 
the beam (I) will be different. The ratio of the beam intensity with and without an 
object in the beam is usually referred to as transmission. 

Two types of interaction with matter are commonly used as contrast modes 
in standard (attenuation-contrast) X-ray (Figure 7.7) tomography of marine life: 
photoelectric absorption and Compton scattering (Figure 7.5). In the case of 
photoelectric absorption, the entire energy of the interacting X-ray photon is used 
to knock an electron out of the shell of an atom of the object and the photon is 
totally absorbed (Figure 7.7a). With Compton scattering, the energy of the photon 
is considerably higher than the binding energy of the electron and an interaction 
will result in a loss of energy of the photon and cause a change in direction. 
Both effects cause the beam intensity to diminish in the forward direction when 
the photons are travelling through the object. Whether a photon will interact is a 
statistical process; therefore, a relatively large number of photons have to be sent 
through the object to measure a correct transmission value. 

The probability of such an interaction depends on the energy of the photon, the 
thickness of the object (I), and the composition of the object and its density. In 
the following we assume a monochromatic X-ray beam of energy E with incident 
intensity f^, and an attenuated intensity I after passing through the object. For a 
homogeneous material, the Beer-Lambert law expresses the exponential decay of 
the intensity 7 as a function of the material thickness L multiplied by its linear 
attenuation coefficient fi. For an inhomogeneous object, the Beer-Lambert law 
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Figure 7.7 Summary illustrating the differ- 
ent types of X-ray interactions, (a) Photo- 
electric absorption results in total removal 
of the incident X-ray photon with energy 
greater than binding energy of electron in its 
shell, with the excess energy distributed to 
kinetic energy of photoelectron, (b) X-ray flu- 
orescence: excitation of an atom by an X-ray 
photon results in decay of the excited state 
by either X-ray fluorescence or the Auger 



effect. K, L, and M are the core electron 
levels, (c) Compton scattering interactions 
occur with essentially unbound electrons, 
with the transfer of energy shared between 
recoil electron and scattered photon, (d) 
Rayleigh scattering is an interaction with 
an electron (or a whole atom) in which no 
energy is exchanged and the incident X-ray 
energy equals the scattered X-ray energy with 
a change in direction. 



can be generalized to: 



I = Iq exp 




ds 



( 7 . 1 ) 



where denotes the map of linear attenuation coefficients for energy E at each 

point {x,y), and D is the straight line joining the X-ray source to the detector pixel 
under consideration. 

The mass attenuation coefficient, defined by the ratio fu, /p, where p is the density 
of the material, is related to the atomic number of the different elements in a 
material. The higher the atomic number of the elements of the material, the higher 
the mass attenuation coefficient and therefore the more probable that an X-ray 
photon will interact with the material. This mass attenuation coefficient varies for 
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different materials and also varies with the photon energy. Generally, the higher 
the photon energy the higher, the chance it will travel through the object without 
interacting. 

7.3.5 

Tomographic Reconstruction 

At a given detector point, the measured transmitted intensity I gives access to the 
integral over the line D of the unknown image function p., known as a projection 
value for this detector point: 



The organization of the projechons values depends upon the acquisition system, 
which has evolved from the 2D parallel or fan-beam geometries to the 3D parallel 
or cone beam geometry (Figure 7.4) (Kalender, 2005). 

Tomographic reconstruction consists of processing all measurements to recon- 
struct the digital image of the object. Owing to the nature of these data, this inverse 
problem is known as image reconstruction from projections (Herman, 1980). 

Image reconstruction methods were first developed for 2D translation-rotation 
CT scanners, historically proposed by Hounsfield (1973, Figure 4a). This geometry 
is known as a 2D parallel beam geometry since for a given rotation angle all 
projection rays are parallel. In this case, the problem is equivalent to inverting 
the Radon transform of the image; it was solved in a purely theoretical context 
by the Austrian mathematician Johann Radon long before the emergence of CT 
scanners (Radon, 1917). In practice today, there are basically two broad classes 
of reconstruction methods: Fourier (and series expansion) methods and algebraic 
methods (Herman, 1980; Kak and Slaney, 1988). Fourier methods are based on a 
continuous formulation of the problem where the image is expressed as a function 
of its projections by an analytic inversion formula. In algebraic methods, the image 
is obtained through the solution of a large linear system, which, given the size of the 
problem, is solved iteratively. While the historical method proposed by Hounsfield 
was an algebraic method called ART (algebraic reconstruction technique), analytic 
methods based on filtered back-projection {FBP) are currently more popular and 
effectively used in all commercial systems. The analytic reconstruction algorithms 
are strongly dependent on the acquisition geometry. Below, we detail FBP in the 
simple 2D parallel geometry and discuss its extension to 3D parallel and cone beam 
geometries. 

7. 3. 5.1 2D Filtered Back-Projection 

Expression of 2D Parallel Projection Let/(x,y) be the image to be reconstructed 
[e.g., the function p(x,y) introduced for the linear attenuation coefficient above]. 
The parallel projection pg (r) at angle 6 and at detector position r (Figure 7.4) is 
the line integral of f(x,y) over the straight line of direction (— sinS, cos0) passing 




(7.2) 



D 
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through the point (rcosO, rsinfi). Thus it can be expressed as: 

+00 

p.(r)=//(rcose-„i„».rs.„« + „ose|d, 

— oo 

The set of projections when 0 and r varies can be represented as a 2D image (the 
Radon transform), which is also called a sinogram in CT. Owing to symmetries in 
the Radon domain, it is sufficient to acquire projections for 9 between 0 and n. 
Figure 7.4f,g illustrates a synthetic digital image and its sinogram. The vertical axis 
corresponds to the rotation angle between 0 and n and the horizontal axis to the 
detector position. 

Fourier Slice Theorem Image reconstruction consists in recovering /(x,yj from 
its sinogram. A fundamental theorem in CT reconstruction is the Fourier slice 
theorem. It states that the ID Fourier transform of pg{r) is a section profile of the 
2D Fourier transform of the image along the straight line in direction 6. Thus, 
when 6 spans the interval [0, jt[, the entire 2D Fourier transform of the image is 
filled, showing that measured data are necessary and sufficient to reconstruct the 
image. Although the Fourier slice theorem can be used directly for reconstruction, 
methods based on back-projection are often preferred. 

Filtered Back-Projection (FBP) The back-projection B^(x, y) of all projections pg{r) 
is the 2D image defined by: 

n 

Bp(x, y) = j pg(xcos9 +ysm6)d0 
0 

Mathematically, the back-projection operator is the dual of the projection operator. 
From a geometrical point of view, the back-projection of a given projection 
“spreads” the values of the projection back onto the reconstruction image 
plane along the projection direction. The total back-projection is the sum of 
the back-projections of each projection. If the projections are filtered with a 
suitable filter before back-projection, the image is exactly recovered. This is the 
filtered back-projection method, which can be written as (Ramachandran and 
Lakshminarayanan, 1971; Shepp and Logan, 1974): 

n 

f{x,y)= pg (x cos 0 + y sin 9)d9 
0 

Pe(r) = {Pe* h){r) 

where * denotes the convolution operator and h is called the reconstruction filter. If 
the bandwidth of the projection is within the interval [—W,W], then the Fourier 
transform of the reconstruction filter h should satisfy the following property: 



F^h(R) = |R| for R e [-W, W\ 



(7.3) 
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where Fj denotes the ID Fourier Transform operator. This corresponds to the 
so-called ramp filter. 

Figure 7.4h illustrates the result of FBP for the synthetic 256 x 256 “fish” image 
when 400 projections between 0 and 180° are used. These reconstructions provide a 
good estimate of the original image. Figure 7.4i shows the result of back-projection 
directly applied to projections without filtering. In this case, a blurred version of the 
image is obtained, a result that can be demonstrated mathematically. Thus, filtering 
is mandatory for reconstruction of the original image. Filtering introduces negative 
values in the projections, which are compensated once all FBPs are combined. 
Filtering can be implemented as a convolution in object space or a multiplication 
in frequency space. In this latter case, the projection is first zero padded to avoid 
aliasing. FBP has been presented here for the 2D parallel geometry, but it can be 
generalized to the 2D fan beam geometry (Herman, 1980). 

7. 3. 5. 2 Image Quality and Artifacts 

Achieving good results with FBP assumes that the sinogram is well sampled 
in terms of angles and spatial resolution. The detector sampling Ar limits the 
bandwidth of the reconstructed image to a disk of radius 1/(2 Ar) in the Fourier 
domain. The angular sampling must be sufficient in the [0, ir] interval. The rule 
of thumb is to take (7t/2)P projection angles if P is the number of samples in the 
detector. When the number of projections is insufficient, streak artifacts appear in 
the image (cf. Figure 7.4j). If the total angle of view is smaller than [0, k], the image 
cannot be recovered exactly (cf. Figure 7.4k). 

The reconstruction filter is another important parameter in FBP. The ideal 
ramp filter defined above is also known as the Ram-Lak filter, an abbreviation 
of the authors who introduced it (Ramachandran and Lakshminarayanan, 1971). 
In the presence of noisy projection data, the ramp filter will amplify the noise in 
the filtered projections and therefore in the reconstructed image. To improve the 
SNR in the reconstructed image, other filters obtained by multiplication with a 
smoothing window in the frequency domain are often used. The Shepp- Logan 
filter uses a sine window but other filters such as Hamming or Hann windows 
have been used as well (Shepp and Logan, 1974). Note that the improvement in 
the SNR is obtained at the cost of a loss in spatial resolution in the reconstructed 
image. 

Whatever the method, image reconstruction requires precise knowledge of the 
position of the axis of rotation axis and of the angular step. Thus, the acquisition 
geometry has to be accurately calibrated with a subpixel precision. Moreover, the 
object must not move during the scan, otherwise motion artifacts appear, leading 
to inconsistent data. 

Exact CT reconstruction also requires the object to be completely included 
in the field of view of the detector. If this is not the case, the projections are 
truncated, which compromises the accuracy of the reconstructed image. In the 
case of truncated projections, also known as local tomography or region-of-interest 
(ROI) tomography, it is possible to use additional information about the object to 
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improve the reconstruction, or to reconstruct a high frequency version of the image 
(Natterer, 1986). 

Other typical CT artifacts are ring artifacts, which appear as rings in the 
reconstructed image, due to imperfect flat-field correction, and beam hardening 
artifacts related to the polychromaticity of the beam, which generates nonlinearities 
in the direct problem that are not modeled by the Radon transform. 

7. 3. 5. 3 3D Image Reconstruction 

3D Parallel Beam CT In the 3D parallel beam geometry, the reconstruction problem 
is straightforward since it can be implemented by the 2D FBP algorithm, which can 
be sequentially applied to the different parallel transverse sections of the 3D image. 

This task is particularly straightforward and can be dramatically speeded up by 
parallel computations (Chilingaryan, 2011) if the axis of rotation, beam direction, 
and detector rows are aligned mutually perpendicular. Thus, the great advantage 
of this geometry is that it is associated to fast and exact reconstruction algorithms. 

3D Cone Beam Geometry Cone beam CT reconstruction algorithms are more 
complex because the information relative to a single slice is distributed along 
different detector lines. Thus, it is not possible to perform a sequence of 2D 
tomographic reconstructions; specific reconstruction methods adapted cone-beam 
projections have to be developed (Herman, 1980; Tuy, 1983; Peyrin, 1985; Grangeat, 
1991). Tuy defined a sufficiency condition for exact cone-beam reconstruction 
(Tuy, 1983): it states that a 3D image is completely determined by its cone-beam 
pro) ections if any plane that intersects the image also intersects the source traj ectory . 

This condition is satisfied by a spiral but not by a circular source trajectory. Thus 
a cone-beam acquisition using a circular trajectory does not uniquely determine 
the 3D image. This non-uniqueness can be interpreted in terms of missing data 
in the 3D Radon space (defined as integrals of the 3D image on 2D planes). As in 
the 2D case, there is an inversion formula expressing the 3D image as a function 
of its 3D Radon transform. But the 3D Radon transform cannot be completely 
sampled from the cone-beam projections when the source describes a circular path. 
Hence, reconstructed images from circular cone-beam data are always approximate 
whatever method is used. Despite this limitation, several cone-beam reconstruction 
methods have been proposed. 

The most popular approach is the Feldkamp algorithm (FDK), which is a 
generalization of the FBP algorithm for cone-beam geometry (Feldkamp, Davis, 
and Kress, 1984). It consists in three steps: weighting of the cone-beam projections, 
filtering of each row of the 2D projections with the usual ramp filter, and cone-beam 
back-projection of the 2D filtered projection. Unlike the 2D FBP, this formula is not 
exact, except for the central plane (z = 0) where the rotation axis is perpendicular 
to the beam direction. The reconstruction errors increase with the distance to the 
central plane. Cone-beam artifacts are more visible on high-contrast objects and 
manifest themselves in the form of geometric distortions and intensity attenuations 
in the vertical direction. Note that cone beam artifacts increase with the cone-beam 
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angle. For this reason, the useful part of the data is sometimes limited to the 
central area of the detector. Many heuristic methods have been proposed to 
correct cone-beam artifacts in circular cone beam geometry (Valton, Peyrin, and 
Sappey-Marinier, 2006). 

Exact cone-beam reconstruction algorithms can be derived if the source describes 
a spiral trajectory. In particular, Katsevich proposed various inversion formulas, 
which can be expressed in terms of EBP (Katsevich, 2004). However, in practice 
these algorithms are rarely used. 

EBP and its variant are the methods currently used in micro-CT scanners 
due to their simplicity and speed. They provide good image quality given the 
limitations and preconditions discussed above. While iterative algebraic methods 
are more demanding in terms of memory and computing time, they are currently 
regaining interest with the advance in high-performance computations. They are 
more flexible in terms of tackling incomplete data sets and they allow the inclusion 
of prior knowledge of the object. They thus offer new perspectives in some 
applications, such as low dose CT with a small number of projections. 



7.4 

Synchrotron laminography 
7.4.1 

Introduction 

Synchrotron-radiation computed laminography (SR-CL) is a non-destructive tech- 
nique for 3D imaging of flat, laterally extended specimens (Helfen et al., 2005). 
It was first developed for inspection of microsystem devices or electronic circuit 
boards (Helfen et al., 2006b) but has now spread to other applications, like materi- 
als science (Moffat et al., 2010; Morgeneyer et al., 2011), cultural heritage studies 
(Krug et al., 2008), and paleontology (Houssaye et al., 2011), where flat, plate-like 
objects are prevalent. The technique is conceptually similar to SR-CT in the sense 
that 3D images are reconstructed from 2D projection data of the specimen. In 
comparison to SR-CT, which is particularly adapted to 3D imaging of specimens 
that are elongated in one direction (e.g., match stick or rod-like samples), SR-CL 
allows the specimens to be extended in two dimensions (e.g., plate-like objects). 

Some basic principles of laminography were already proposed in the 1920s to 
1930s by Bocage and Ziedses des Plantes, who independently worked on early con- 
cepts (Bocage, 1922; Ziedses des Plantes, 1932) in which 2D cross-sectional slices 
(from one to a few) could be imaged by a single scan. It involved employing a syn- 
chronized motion of a movable X-ray source and 2D detector (film) during exposure. 
The geometry of this synchronous motion defines one so-called focal plane, which 
appears sharply imaged. The greater the distance towards this focal plane, the more 
blurred the object planes away from this focal plane are rendered. Up until the 1960s 
this concept was steadily refined toward a technique called tomosynthesis (Grant, 
1972). By acquisition of several radiographs during the scanning motion any given 
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plane could be optically reconstructed from a single scan. With the development of 
CT (Cormack, 1963; Hounsfield, 1972) during the 1960s/1970s, medical imaging 
researchers gradually abandoned this technique, which, however, remained in use 
for industrial applications (e.g., for device inspection). In industry it was retained 
as it enabled flat, and laterally extended, components (like printed circuit boards) to 
be investigated non-destructively (Zhou et al., 1996; Kalukin and Sankaran, 1997; 
Gondrom et al., 1999). Owing to the widespread use of digital imaging detectors 
(such as flat panel detectors or CCD-based X-ray detectors) and to increasing com- 
puting power, these methods (nowadays called digital tomosynthesis and computed 
laminography) have regained strong interest (Dobbins and Godfrey, 2003). 

The benefits of using synchrotron-generated X-rays instead of desktop-generated 
X-rays in laminographic imaging techniques are similar to the case of CT. As an 
example, monochromatic radiation increases the image contrast due to avoidance of 
beam hardening artifacts. Element-specific imaging can be performed by working 
in the vicinity of absorption edges of specific chemical elements, as described in 
the following sections. Moreover, fast acquisition times at high spatial resolutions 
can be realized due to the high photon flux density. Finally, the method can be 
sensitive towards the electron density via various phase-contrast methods (Nugent, 

2010; see also the following sections). 

Presently, in most synchrotron imaging set-ups the usable beam cross-section is 
rather small, which is well suited for high-resolution imaging but offers a limited 
field of view compared to a conventional laboratory X-ray tube. In synchrotron 
laminography, the source cannot be moved and thus has to be kept stationary 
compared to its laboratory counterparts. Moreover, high-resolution X-ray detectors 
need to be positioned with a precision of a fraction of the effective pixel size during 
the scanning motion. Consequently, SR-CL uses a scanning scheme where both 
X-ray source and detector are stationary and only the specimen is rotated around 
an axis inclined with respect to the beam direction. 

SR-CL can be considered as a generalization of SR-CT (Helfen et al., 2011), 
with a variable inclination angle 6 between the rotation axis and the direction of 
the transmitted beam (Figure 7.8a,b). For CT, this corresponds to 6 = 90°. The 
specimen itself can be laterally (i.e., in the two directions perpendicular to the 
rotation axis) larger than the width of the detector’s field of view but should be 
thinner in the third dimension (i.e., along the rotation axis). If the specimen 
is oriented approximately perpendicular to the rotation axis, the average X-ray 
transmission to the 2D detector does not change significantly during specimen 
rotation. If the inclination angle of the rotation axis is well chosen, reliable 
projection data (i.e., projections with sufficient X-ray transmission and SNR) can 
be acquired for the entire scanning range of 360° around the rotation axis. 

Similarly to all limited-angle scanning schemes, the main drawback of this 
scanning geometry lies in unsampled spatial frequencies in the specimen’s Fourier 
domain, which prohibits accurate reconstruction. These missing frequencies ori- 
ented in a double-cone around the rotation axis (Figure 7.8a,b) give rise to artifacts 
in the reconstructed images that depend on the specimen’s structure (Helfen et al., 
2005). In the case of CT (6 = 90°) the coverage of the Fourier domain is complete 
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Laminography 



Absorption radiograph 



Laminogram 



(c) Dermal denticles of shark 



(d) Mussel shell 



(e) Dermal denticles of shark 




Figure 7.8 Synchrotron laminography (SRCL) with parallel-beam geometry: (a) experimental setup illustrating the acquisition geometry and 
(b) Fourier-domain coverage of SRCL; (c) absorption radiograph, (d,e) laminogram (reconstructed slice) of a mussel shell, Mytilus edulis, and 
dermal denticles of shark, Centroscymnus coelepis, and (f) volume rendering of dermal denticles of a shark. 
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(up to a given resolution limit) and the specimen can be reconstructed faithfully. 

If insufficient transmission is the reason for the inaccessible angles, then these 
missing frequencies have to be accepted for any scanning scheme, however, and 
the given X-ray energy determines the maximum angle at which the specimen 
can be inclined to provide useful projection data. The SR-CL scanning geometry 
maximizes coverage of the specimen’s Fourier domain in these cases. 

7.4.2 

Image Reconstruction 

In analogy to Section 7.3.5, Fourier methods can be derived for analytical 3D image 
reconstruction. A projection is now considered as a 2D image with the additional 
parameter t expressing the height coordinate (defined along the projected rotation 
axis) in the image. The image acquisition geometry has a further parameter i7, 
which is the inclination angle between the rotation axis and the direction of the 
transmitted beam: 

/ OO 

(r^sjfjds (7.4) 

■OO 

where the superscript T denotes the transpose operation, and the coordinate 
transform matrix: 



( COS0 sin0 0 \ 

— sindsinil sin ;7 cost? cos d j (7.5) 

COS I? sin d — cos cos 0 sin & / 

describes a rotation around the axes x and z by the angles 90° — and 9, 
respectively. The angle 9 is varied for 2D data acquisition around the scanning axis 
z. In the case of 3D parallel-beam CT, the axis inclination angle 9 corresponds 
to 90°. 

To calculate the back-projection image Bp(x,y,z), all radiographic projections 
Pg g(r,t) are smeared back in the reconstruction space along the same trajectory as 
the projections were measured initially. More formally, it is defined by: 

Bp(^> y. z) = r «>) • (^, y- z)^]d0 (7.6) 

Jo 

where denotes a 3 x 2 matrix where only the first and the third rows of the 

matrix ^ are retained. 

Again, the Fourier slice theorem can be used to express correspondence between 
the 2D Fourier transform of the projection image and the central slice (i.e., through 
k = 0) through the 3D Fourier domain, whose normal vector has the same direction 
as the projection direction. The principle ofback-projection from Section 7.3.5 can 
now be extended to three dimensions, incorporating this additional parameter 9 
corresponding to the axis inclination. The point-spread function of the 3D back- 
projection for the laminography geometry has been derived by Matsui et al. 1993 
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and can be written in cylindrical coordinates as: 






H : fej, > |fe^| ■ cot!? 

0 : elsewhere 



( 7 . 7 ) 



where ^ fej + k^,kg= arctan(fej,/fej^) , and i = {k^,k^,k^)^ is the Fourier conjugate 

to the real-space vector {x,y,z)^ of a coordinate system fixed to the specimen. 

As mentioned before, we see that there are vacancies with the shape of two 
cones along the positive and negative k^ directions in the 3D Fourier domain. 
In these regions, information about the specimen is missing completely, and 
therefore image reconstruction can only be approximated. This also explains why 
in laminography the reconstructions are more prone to artifacts than in tomography 
reconstructions . 

Defining the (approximate) 3D inverse filter function (Lauritsch and ITarer 1999; 
Helfen et al. 2006a): 



H(kp,kg,k^) = 



[^^kl-kl-col^Y' : kp > |kj . cot!? 

0 : elsewhere 



(7.8) 



the unknown image function can be estimated using the reconstruction equation: 



f{x,y,z) = [h*Bp}{x,y,z) (7.9) 

where h = and * again denotes convolution. 

As for 2D tomography, a filter function can be derived which can be employed 
with the back-projection operation introduced above and serves to dampen the 
overrepresented low spatial frequencies in the back-projected 3D image while 
retaining as much information as possible from the specimen. 

Using the filter function (Helfen et al, 2011): 

- — , . sin !? 

= — ■ |fe,| (7.10) 

filtering is then performed for all 2D projections by the relationship: 






(7.11) 



prior to back-projection. 

We see that projection filtering is the same for all projections acquired, that is, 
there is no dependency on the scanning angle 0. Furthermore, it does not depend 
on the direction t parallel to the rotation axis (projected to the detector plane) 
and it can be performed by a procedure similar - apart from the (constant) factor 
(siniJ)/2 - to the ID filtering known from CT. 

SR-CL allows imaging of relatively small regions from laterally large objects with 
high spatial resolution, that is, only a small ROI defined by the detector’s field of 
view is reconstructed around the crossing point of beam axis, rotation axis, and 
specimen. The field of view of the detector along the rotation axis then determines 
the maximum specimen thickness. 
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In the following, we outline the interest of SR-CL for high-resolution cross- 
sectional or 3D imaging of specimens typical in marine biology. As in other 
application fields, employing phase-contrast methods to improve contrast condi- 
tions is a substantial benefit to laminographic imaging (Helfen et al., 2009; Xu 
et al., 2010; Altapova et al., 2012). In comparison to SR-CT, the interest in laminog- 
raphy lies in its abilities to achieve high spatial resolutions for laterally extended 
specimens that are too precious to be cut down to the size required to perform a 
CT scan, for example, museum objects, fossils, or extinct species. 

As for SR-CT, the specimens should be stable enough to withstand the irradiation 
by X-rays without deformation on the micrometer scale during the scanning. 
Otherwise, motion artifacts would occur. This is especially important for soft 
tissues (e.g., skins), which may need to be embedded into a resin of another 
supporting matrix to be stabilized. 

7.4.3 

Example Applications 

To illustrate the potential of our approach, we applied SR-CL to image the skin 
structure of a shark and the microstructure of shells. Figure 7.8d shows a recon- 
structed slice through the microstructure of a mussel shell {Mytilus edulis). A 
mussel shell is the product of biomineralization processes. Biominerals are com- 
posite materials made up of inorganic and organic components, and show unique 
morphologies. However, climate change is predicted to affect biomineralization 
processes in many marine organisms and thus scientists seek to understand how 
these processes may be affected by biotic and abiotic factors. Figure 7.8c,e,f shows a 
radiograph, a reconstructed slice, and a 3D rendering of shark skin. Both scientists 
and engineers have long suspected that the denticles on the skin of sharks reduce 
hydrodynamic drag during locomotion. In a recent study Oeffner and Lauder 2012 
demonstrated the hydrodynamic nature of the denticles on the skin of sharks and 
developed two biomimetic applications. 



7.5 

Absorption Imaging 
7.5.1 

Natural Contrast 

X-Ray p,CT (Baruchel et al., 2008; Peyrin, 2009) is based on absorption, that is, the 
contrast in attenuation between different tissues. In conventional X-ray imaging, 
the observed contrast results from the variation in photon absorption arising from 
density and composition differences between different tissues. The technique yields 
excellent results when a highly absorbing structure is embedded in a matrix of 
relatively weak absorbance (e.g., bone inside a body or shells) (Stock et al., 2003; 
Prymak et al., 2005; Perez-Huerta et al., 2009; Pasco-Viel et al., 2010). Bones and 
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shells have both a higher density and a chemical composition with a higher mass 
attenuation coefficient than soft tissue, such as muscles or fat, and are therefore 
very easy to discern. As the soft tissues of marine organism are predominantly 
composed of water, it is very hard to distinguish soft tissue structures from one 
another. In some cases, it is possible to distinguish water-filled volumes from 
muscle and fatty tissues, but these differences are rarely sufficient to study the 
morphology of these organisms. As illustrated in Figures 7.9 and 7.10, a standard 
CT-scan of a fish reveals bones but soft tissue information is absent. 

7.5.2 

Staining Contrast 

When the contrast is insufficient to study soft tissue morphology, it is possible 
to use staining agents. These agents are very similar to those used for staining 
histological samples in optical, fluorescence, and electron microscopy (Betz et al, 
2007; Metscher, 2009). 




Figure 7.9 Two types of contrast formation 
available in absorption X-ray microtomog- 
raphy, that is, natural contrast (a) versus 
staining contrast (b). This plate illustrates 
three cross-sectional slices (sagittal, trans- 
verse, and coronal) of a golden ide {Leuciscus 
idus) scanned with a desktop |xCT for consis- 
tency. (a) The set of slices of a fish with no 



staining. Bones can be distinguished from 
the apparently homogeneous soft tissues. 

(b) The second set of slices shows another 
specimen of the same species offish but 
stained with phosphor-molybdenic acid. The 
morphological features of soft tissue are now 
accessible. 
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These agents typically have a much higher mass attenuation coefficient due to 
the presence of elements with a high atomic number, such as zinc, bromine, 
molybdenum, silver, iodine, barium, gadolinium, tungsten, osmium, gold, mer- 
cury, and lead. This results in a higher absorption of the structures that contain the 
agent and therefore a better contrast between the different structures (Figure 7.9), 
which, in turn, allows. This allows the visualization of soft tissues, such as nerves 
(Johnson et al., 2006), muscles (Johnson et al, 2006; Betz et al., 2007), or vascular 
system (Plouraboue et al., 2004; Heinzer et al., 2006; Wirkner and Prendini, 2007; 
see also Figure 7.10d). 

In addition to the enhanced contrast, special imaging methods, such as K-edge 
digital subtraction imaging (KES) can be applied to improve image quality and 
structure discrimination, as the contrast agents contain one specific chemical 
element not normally present in the organisms (Figure 7.11). The K-edge of a 
chemical element describes a sudden increase in the attenuation coefficient of 
photons occurring at a photon energy just above the binding energy of the K shell 
electrons of the atoms interacting with the photons (Adam et al., 2005). 

To perform K-edge imaging, two images are recorded, ideally simultaneously (to 
avoid motion artifacts on images), using monochromatic beams with energies that 
are below and above the K-edge of the chemical element used as contrast agent 
(Figure 7.11). 

A simple algorithm allows retrieval of the distribution map of the contrast agent 
(Esteve et al., 2002; Sarnelli et al., 2005). Using SR, the technique has been applied 
in several biological fields, including bronchography (Bayat et al., 2001), coronary 
angiography (Esteve etal., 2002; Sarnelli etal., 2005), brain vessel imaging (Schultke 
et al., 2010), and high atomic number (Z) based nanoparticles metabolization (Alric 
et al., 2008). 

The development and application of the KES technique are closely related to 
the scarce availability of suitable SR sources and the engineering of compact 
dual-energy sources coupled with adapted detectors. Compact sources tailored for 
pre-clinical and clinical application of the dual-energy technique are described 
in the literature (Baldelli et al. 2005). In these sources, the beam generated by a 
conventional X-ray tube is monochromatized (down to a few keV energy bandwidth) 
by highly oriented pyrolytic graphite mosaic crystals. Two thin adjacent parallel 
beams of energies bracketing the K-edge of iodine, 31.2 and 35.6 keV, respectively 
(energy resolution <10%), are obtained by means of a proper collimation system. 
Unfortunately, these promising prototypes have not yet reached widespread use. 





Figure 7.10 X-Ray microtomography with a 
parallel-beam geometry; experimental setups 
of (a) absorption tomography and (e) phase- 
contrast tomography. (b) Radiography and (c) 
virtual slice of Osmerus eperlanus in sagittal 
view at head level, (d) Thermocyciops consimilis 
sagittal radiography, (f) Lateral radiography and 
(g) coronal virtual slice of Laticauda saintgironsi, 



a sea snake from New Caledonia (flot Brosse, 
November 2003; MNHN 2009.0168 Coll. Ivan 
Ineich), at head level. Sagittal slice of (h) Argy- 
ropeiecus acuieatus and of (i) Branchiostoma 
platae - view of anterior part of animal. Note 
that in phase contrast the black and white 
fringes represent interfaces between tissues. 
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Figure 7.11 K-edge subtraction (KES) imag- the compound barium suifate). The abso- 
ing. In KES imaging, two CT images are lute quantity of the contrast agent at any 

acquired using X-ray beams at two different given point of the CT image (a vessel in this 

energies, above and below the K-edge of a example) is determined by subtracting the 

given chemical element (in this case Ba in two images on a logarithmic scale. 

Alternative K-edge imaging and dual energy techniques are being developed for 
polychromatic sources typically found in laboratory setups and medical scanners. 
These methods consist of comparing the reconstructed attenuation coefficient of 
a voxel for scans performed with a different X-ray spectrum. The variation of 
this coefficient is related to its energy-dependency, which can be calculated with a 
system of equations. This approach has proven to be very successful and is currently 
being implemented in the latest medical CT scanners and laboratory setups. 



7.6 

Phase-Contrast Imaging 
7 . 6.1 

Introduction 

X-Ray PCI enables mapping of the variations in the X-ray refractive index of 
the object under study and, thereby, of its electron density. Unlike absorption 
contrast, phase contrast is generated by the interference of the diffracted or 
refracted components of the beam with each other. At the detection plane, these 
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interferences form intensity variations, that is, an X-ray image that can be recorded, 
usually with the same type of detector as an absorption-contrast image. Phase 
contrast takes its name from the fact that it is not only created by differences in 
the amplitude of the X-ray wave after it has passed through the object but also by 
variations of its phase (Figure 7.6). 

The great advantage of PCI is that it can be up to 1000 times stronger than 
absorption contrast for hard X-rays, especially when the object investigated consists 
largely of elements with low atomic number 2 (as is the case, for example, for 
biological soft tissue), or when the absorption differences within the sample are 
weak (as in many fossilized organisms). 

In addition, most phase-contrast techniques are differential, that is, the images 
obtained with these techniques represent a spatial derivative of the object function’s 
projections, rather than the projection itself. For example, in propagation-based 
phase contrast (Section 7.6.2, below), edges and interfaces in the object are partic- 
ularly enhanced in the images, through a pair of bright/dark fringes (Figures 7.3, 
7.5, 7.6, and 7.10). This improves the visibility of small structures. 

On the other hand, one additional challenge posed by the use of differential 
contrast techniques is that the usual tools for the quantitative analysis of structures 
are adapted to absorption-based CT datasets and that their application to phase- 
contrast datasets is not always possible. 

This problem can be solved by the application of phase retrieval techniques. 
In differential PCI, phase retrieval (also named phase reconstruction) essentially 
consists of obtaining the original object function from the derived images, mea- 
sured by the detector in a differential phase-contrast (DPC) measurement, and in 
separating the information on the refractive index from that on the absorption 
coefficient. These two contributions are usually superimposed on each other in 
the raw detector images. Such techniques are also called quantitative phase-contrast 
techniques or simply phase imaging (as opposed to PCI). In general, quantitative 
phase imaging either requires more data than a simple absorption-contrast data 
set (Cloetens et al., 1999; Weitkamp et al., 2005a) or it requires ad hoc assumptions 
on the sample composition (Nugent et al., 1996; Paganin et al., 2002; Groso et al., 
2006), otherwise it will result in an image with inferior spatial resolution than 
the detector would allow for absorption-contrast images. This latter phenomenon 
occurs, for example, when moire techniques are used (Weitkamp et al., 2005b; 
Momose et al., 2011). 

Among the different X-ray phase-contrast techniques used today, some require 
additional X-ray optical elements in the beam, while others do not. In addition, 
depending on the method, the quantitative phase recovery process may be very 
simple and highly-automated, or more complicated and require careful manual 
intervention. 

A requirement common to all phase-contrast techniques is that of a sufficient 
degree of lateral coherence, and sometimes also longitudinal coherence, of the 
X-ray beam. Lateral coherence can simply be expressed in terms of the size at 
which the X-ray source appears when seen from the position of the sample: if 
the source has a lateral size s and is located at a distance L from the sample its 
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angular size is approximately s/I (if there are no focusing or scattering optical 
elements in the beam between source and sample). The lateral coherence of the 
beam becomes stronger as the angular source size decreases, and it is often 
expressed in terms of the lateral coherence length XL/s (where X is the X-ray 
wavelength). Since the source size can be different in the two transverse directions 
X and y (i.e., orthogonal to the beam propagation direction z), so can the lateral 
coherence length. Longitudinal coherence, on the other hand, is simply related to 
the monochromaticity of the X-rays used: the narrower the X-ray spectrum, the 
higher the longitudinal coherence. 

Although we present here phase-contrast data taken at a third-generation syn- 
chrotron (ESRF), which delivers X-rays with a high degree of lateral (and, when 
needed, longitudinal) coherence, it is important to point out that monochromaticity 
and spatial coherence requirements for qualitative PCI are minimal if the required 
sensitivity is only moderate (Margaritondo andTromba, 1999). 

In fact. X-ray PCI has been used at second-generation synchrotrons, especially 
for medical and materials science applications (Momose et al., 1996; Beckmann 
et al, 1997; Castelli et al, 2011; Tsai et al, 2002; Muehleman et al, 2004), and even 
with conventional X-ray sources (Wilkins et al, 1996; Zhang et al, 2008). In the 
following, we describe in more detail three different classes of X-ray PCI: free-space 
propagation, analyzer-based imaging (ABI), and grating interferometry. 



Free-Space Propagation Methods (Holotomography) 

If a sufficiently coherent X-ray beam is used, a particularly simple mode of PCI 
becomes available. By simply letting the X-ray beam propagate in free space 
after interaction with the object, phase contrast is achieved (Snigirev et al, 1995). 
The contrast formation process can be understood in the framework of Fresnel 
diffraction. When an X-ray beam passes through an object, it may be affected in two 
ways: it might be absorbed in the object, which changes its amplitude, and it might 
be slowed down in the object, which changes its phase (Figures 7.10 and 7.12). 

Amplitude and phase modulation are, respectively, related to the imaginary and 
the real part of the complex-valued X-ray refractive index: 



where is the refractive index decrement, fi is the attenuation index, i is the 
imaginary unit number, and (x,y,z) are the spatial coordinates . Both the attenuation 
and the phase shift induced by the object can be described as projections through 
the absorption and refractive index distributions, respectively, with the amplitude: 



7.6.2 



n{x, y,z) = l- 5„(%, y, z) + ifi{x, y, z) 



(7.12) 




(7.13) 



and phase: 




(7.14) 
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This means that both parts of the complex refractive index can be reconstructed by 
tomographic reconstruction if amplitude and phase can be measured for different 
angular settings of the sample. The wave field at the exit plane of the object, 
assuming a fiat incident beam, is then: 

T(x) = exp[-B(x) + i^(x)] (7-15) 

For the relatively short propagation distances we are interested in here, usually 
up to 1 m, the effect of propagation on the exit wave-field can be described in 
the framework of Fresnel diffraction (Goodman, 2005). The intensity f£,(x) at 
propagation distance D downstream of the object is then written as: 

/b(x) = |Fro[T](x)|2 = |T(x) * (7.16) 

where Fr^, denotes the Fresnel transform, T(x) is the wave field at the exit of 
the sample, and P£,(x) is known as the Fresnel propagator. We can see that the 
relationship between phase shift and wave field at distance D is linear, while the 
relationship between phase and contrast is nonlinear. If,(x) is the quantity that can 
be detected and is called a phase-contrast image or a Fresnel dijfraction pattern. These 
images can be used directly as input for tomographic reconstruction algorithms. 

The reconstructed image then shows an edge enhancement effect that is related to 
the Laplacian of the 3D refractive index (Cloetens et al., 1997), which can be a useful 
modality in its own right. More interestingly, the phase shift can be reconstructed 
from a series of phase-contrast images at different distances through a process 
known as phase retrieval. This is not straightforward due to the nonlinearity in 
Eq. 7.15. Usually, this problem is addressed by linearization of Eq. 7.16 (Nugent 
et al., 1996; Cloetens et al., 1999). Several approaches have been developed. A 
frequently-used method is based on linearization with respect to the object and 
yields the linear contrast model (Guigay et al., 2007): 



7MOD(f) + 2sin(7tXD|f|2)F(lo^l(f) +cos(7iTD|f|2) — F{V • (^Vlo))(f) 

ZTt 

(7.17) 

where/(f) = F{fj(f) denotes the 2D Fourier transform of/(x), which is valid under 
slow variation conditions on the attenuation and phase. The phase is retrieved by 
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Figure 7.12 Holotomography. (a) Acquisition 
geometry: The imaging setup of a synchrotron 
radiation propagation-based phase-contrast 
imaging system. X-rays are taken from an inser- 
tion device, and then monochromatized. The 
long source-to-sample distance yields a high 
degree of spatial coherence. The sample is 
mounted on a translation-rotation stage (stan- 
dard SR-jiCT setup). The detector, consisting 
of a scintillator, light microscope optics, and a 
CCD, is mounted on a translation stage to allow 



for free space propagation of the beam after the 
sample, (b) Radiographies of Osmerus eper- 
lonus head, from dl to d3, show the increase in 
phase contrast (see at interface of bubble), (c) 
Phase retrieval map. (d) Virtual slice obtain by 
holotomography showing an artifact induced 
by free bubble. The contrast is affected by the 
presence of bubble, (d) Virtual slice obtain 
by holotomography without free bubble. Note 
that the slices are windowed to obtain the best 
contrast in soft tissues. 
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solving the linear least squares minimization problem: 



argmin V ||7^oO(f) + a||^(f)||' 

mlf\ • * 



where are the images recorded at different distances, typically 2-4. The 

second term is a regularization term and is usually classic quadratic Tikhonov 
regularization with ip^ = 0. Owing to the low sensitivity of the contrast formation 
mechanism to low frequencies in the phase (which can produce low frequency 
noise in the retrieved phases), it is sometimes desirable to introduce other a priori 
knowledge in the regularization term; for example, homogeneity of the object 
(hanger, Cloetens, and Peyrin, 2009, 2010). In the particular case of an object 
made of a single material with known X-ray optical properties, in which only the 
density of the material shows spatial variations but its chemical composition does 
not, the phase profile of the wave front at the exit plane of the wave front can 
be retrieved with a particularly simple, non-iterative equation known as Paganin’s 
formula, namely, (Paganin et al., 2002; Weitkamp et al., 2011): 



While the condition of uniform chemical composition is rarely fully met, Paganin’s 
method is, in practice, quite tolerant to deviations from ideal conditions (Weitkamp 
et al., 2011). This robustness, combined with the fact that the formula is easy to 
implement and allows reconstruction from a data set taken at a single sample- 
to-detector distance (just as in absorphon-contrast tomography), makes Paganin’s 
method a very versatile reconstruction method for parallel-beam, phase-contrast 
data in free-space propagation phase contrast. 

7.6.3 

Analyzer-Based Imaging 

Analyzer-based imaging (ABI) is a phase-contrast technique based on the registra- 
tion of a monochromatic and parallel radiation transmitted through an object, after 
having been analyzed by a perfect crystal. When the beam transmitted through 
the object falls on a perfect silicon crystal, the latter acts as a very narrow angular 
slit, reflecting only the rays that make the correct angle with the crystal surface. By 
tuning the crystal, the non-deviated, refracted, and scattered rays can be separated 
on images. In a typical case, the full-width at half-maximum of the reflectivity curve 
of the analyzer is a few microradians, and its shape is approximately triangular 
with linear slopes. Figure 7.13 shows a typical ABI setup. The analyzer is tuned to 
reflect at the angle that corresponds to the peak of the intrinsic (without an object) 
reflectivity curve (top), or at the mid-slope on either side of the reflectivity curve 
(low or high). At the top, all scattered and refracted rays are rejected, leading to the 
so-called apparent absorption contrast (extinction), while at low and high positions, 
the changes in the propagation direction of X-rays are converted into intensity 
changes giving rise to the (so-called) refraction contrast. ABI (also called diffraction 
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Figure 7.13 Analyzer-based imaging (ABI). 
(a) Schematic diagram of the ABI set-up: a 
narrowband monochromator using Si(333) 
reflections, analyzer Si{333), and the detec- 
tor. The sample can be scanned vertically 
through the sample [radiography, e.g., L, H 
in part (b)] or rotated (tomography). The 
ionization chambers are used to tune the 
system, (b) Inset: reflectivity curve and the 
principle of the ABI method. The reflectiv- 
ity curve, R(s), demonstrates the conver- 
sion between the angular X-ray deviations 
(horizontal axis) and the intensity changes 
(vertical axis). ABI, analyzer-based X-ray 



imaging; P, peak or top position; L, low- 
angle position; H, high-angle position. Radio- 
graphy L and H shows different phase con- 
trast. (c) Computed tomography of a human 
ankle images using the ABI technique; X-ray 
energy 51.5 keV; pixel size 45 mm. (d) Lat- 
eral slices of a human big toe constructed 
by ABI tomography acquisitions (experimen- 
tal parameters: X-ray energy 30 keV, pixel 
size 45 mm. The arrows indicate the edges 
of the cartilage tissue covering the two joint 
bones, (c and d: Reused from Coan et al. 
(2008) with kind permission from Elsevier.) 



enhanced imaging) has been the principal method in PCI of human breast tissue 
specimens. 

7. 6. 3.1 Applications 

Most developments in ABI have been carried out at SR facilities, because presently 
these are the only sources capable of delivering intense monochromatic and 
collimated X-ray beams. 

In mammography, after the exceptional pioneering results obtained, respectively, 
at the Brookhaven National Laboratories (BNL) and at ELETTRA (Trieste, Italy), 
on imaging thin pathological (Pisano et al, 2000) or full healthy breasts (Arfelli 
et al., 2000), research has focused at the ESRF on developing low dose tomography 
imaging (Keyrilainen et al., 2008; Keyrilainen et al., 2010). With the aim of 
overcoming the severe limitations imposed by present clinical X-ray tools, a very 
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promising application of ABI is in osteoarthritis (OA) diagnosis, with the ancillary 
objectives of understanding the evolution of the illness and of non-invasively 
studying the effectiveness of drug treatments. Examples of images of human 
joints acquired with the ABI technique are reported in Figure 7.13c (ankle) and in 
Figure 7.13d (big toe). Milestones of this development are the first radiographic 
detection of the structural orientation in human cartilage (Muehleman et al., 
2004) and the first correlation of the pathological status of a sample with the spatial 
distribution of the chondrons, which are clusters of cartilage cells (Coan et al., 2010). 

Analyzer-based images also allow the identification of the quality of ingrowth of 
bone into the hydroxyapatite layer of an implant; in fact, it has been shown that 
incomplete integration of the implant with a remaining microscopic gap caused 
the presence of a highly refractive edge at the implant/bone. Therefore, ABI could 
be utilized to diagnose implant healing and/or loosening (Wagner et al., 2006). 

7.6.4 

X-Ray Crating Interferometry 
7. 6.4.1 Introduction 

Propagation-based X-ray phase contrast (Section 7.6.2) yields excellent results for 
studies at high spatial resolution (i.e., with pixel sizes in the micron and sub- 
micron range) and using moderate X-ray photon energies. However, for larger 
pixel sizes and higher energies, carrying out a propagation-based measurement 
under optimum conditions becomes more and more difficult. This is because the 
propagation distance for optimum contrast in propagation-based setups, D^^^, scales 
with the square of the pixel size and with the inverse of the X-ray wavelength 
X, that is, Djjpj « (2dpj^)^/2. (Weitkamp et al., 2011). Imaging full objects from 
centimeter size upward will usually require pixel sizes in the range of 10 pm or more. 
Propagation-based phase-contrast images of such objects can still be obtained, but 
the experimental propagation distances often have to remain far below their 
theoretical optimum values, which quickly attain impractical values of dozens of 
meters. As the real distances used during imaging remain much smaller, the phase- 
contrast signal obtained is not as strong as it could be under optimum conditions. 

Therefore, when the required field of view exceeds a centimeter, other phase- 
contrast methods can become advantageous. ABI (Section 7.6.3) is one of these 
techniques. 

Another, more recently developed approach, is X-ray grating interferometry 
(XGI), sometimes also called Talbot interferometry. As well as providing high 
sensitivity in phase-contrast radiography and tomography, XGI has the important 
advantage of being compatible with low-brilliance X-ray sources such as standard 
laboratory X-ray tube generators. Moreover, XGI gives access to an additional 
contrast modality called “dark-field imaging”, in which scattering regions in the 
sample can be visualized. 

Principles In XGI (Momose, 2003; Weitkamp et al., 2005a), a line grating G1 
is placed in the X-ray beam, usually between the sample and the detector 
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(Figure 7.14a). The period pj of the grating lines is much larger than the X- 
ray wavelength k, but often smaller than the detector pixel size. The modulation 
that grating G1 introduces in the X-ray wave front will produce a set of interference 
fringes at positions downstream of Gl. If the incident wave is planar or mildly 
spherical, these fringes will be regular and periodic, with the same or half the period 
of Gl (depending on the thickness of the grating lines), and will have uniform 
contrast over the field of view. However, if an object under study is introduced in 
the beam path, the fringe pattern downstream will be displaced or distorted due 
to refraction of the X-rays by the specimen, and the contrast of the fringes may be 
locally reduced because of scattering in the object. Moreover, as in conventional 
X-ray absorption-contrast imaging, the overall intensity averaged over one or more 
fringe periods will be decreased as a function of X-ray attenuation in the sample. 

The principle of XGI is to detect the distortions and/or contrast variations of the 
fringes created by Gl and modified by the presence of the specimen. Using this 
principle, three complementary images can be retrieved: a DPG image, a visibility 
image (also called a “dark-field image,” DFI), and the attenuation image. However, 
as mentioned above, the period of the grating Gl, and therefore also the period 
of the fringe pattern, is usually too small to be resolved by the detector. This is why 
a second line grating G2 is placed directly in front of the detector. Grating G2 acts 
as a transmission mask for the detector. Its lines should therefore strongly absorb 
the incident X-rays, and the period P 2 of these absorbing lines should be identical 
to the period of the interference fringes generated by Gl. In areas where the 
intensity maxima of the fringe pattern incident on G2 coincide with the positions 
of the absorbing lines, the detector will record low intensity. In areas where the 
fringe maxima lie in the clear spaces between the grating lines, the detector will 
record high intensity. G2 thus encodes fringe position into intensity values on the 
detector. However, in a single interferogram thus recorded the information on 
fringe position and fringe visibility will still be superimposed on one another and 
on the attenuation signal. 

Several approaches exist to separate and access all three types of information. The 
two most common ones are moire imaging and phase stepping (also called “fringe 
scanning”). In moire imaging, gratings Gl and G2 are slightly rotated with respect 
to each other, around the beam axis (Weitkamp et al., 2005b; Wang et al., 2011). 

This results in a pattern of moire lines recorded on the detector. Their analysis 
gives access to the three contrast modalities based on a single image. The spatial 
resolution of this technique is, however, limited to the period of one moire fringe, 
that is, it is inferior to the resolution of the detector itself. In the phase-stepping 
technique, the full spatial resolution of the imaging system is retained. Here, the 
gratings are aligned with their lines parallel to each other, and several images (their 
number N typically ranging between 3 and 5) are recorded for different transverse 
positions of one of the two gratings. The scanned grating is moved transversally 
in steps that are a fraction of the grating period p. The full range of the scan is 
usually one fringe period. The intensity I in each pixel with coordinates x = (%,y) 
will oscillate during the scan, as a function of the grating position = (m/N) x p. 

This oscillation can usually be well approximated by a sine function: 
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Differential phase contrast (DPC) 






Figure 7.14 Grating interferometry with a 
parallel beam geometry. Principle of tomog- 
raphy using X-ray grating interferometry: The 
setup (a) is similar to other microtomog- 
raphy methods but two gratings G1 and 
G2 are introduced between the specimen 
and detector. The sample and (if applica- 
ble) its container are often immersed in 
water (or another phase-matching liquid) 
to avoid artifacts from strong contrast at 
the outer edges. At each projection angle of 



the tomography scan, one of the gratings is 
laterally scanned over one period in a few 
steps. The images recorded from the detec- 
tor (b) are processed to yield three contrast 
modalities (c): a differential phase-contrast 
(DPC) radiograph, a dark-field image, and an 
attenuation image. The specimen is a lizard, 
immersed in water. The two images (d) and 
(e) show an absorption tomogram (d) and a 
phase tomogram (e) of a soft-tissue sample 
(the kidney of a rat). 
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Here, I^ix) is the detector signal recorded in the m-th phase step in the pixel 
coordinate x. A pixel-wise analysis of the N images obtained in a phase stepping 
scan, for example, by Fourier methods, easily yields the parameters a, b, and i/f. 
This procedure is fully automated. The attenuation image A(x), DPC image a(x), 
and DFI v(x) are obtained from these parameters by: 



Attenuation : A(x) = a{x) /a^^{(x) 

DPC : a(x) = {p^/ilnD)] x [i/f(x) - f,^i(x)] 

DFI : u(x) = b(x)a^^i(x)/[b,^i{x)a(x)] 



In the formulas above, the subscript “ref’ indicates reference images taken in the 
beam without a sample. Note that the attenuation image A{x) corresponds to the 
intensity transmission through the sample. The DPC image a{x) is the angle of 
refraction in the direction perpendicular to the grating lines. It is related to the 
wave-front phase shift profile (p(x) by: 



a(x) 



X d(p{x) 
2rr 9x 



Note that the inter-grating distance D in the denominator of the DPC formula 
implies that the sensitivity of the instrument will increase if the distance between the 
gratings is increased. In this way, the phase sensitivity of a grating interferometer 
can be tuned by choosing the distance D between the gratings. 

All three signals - attenuation, DPC, and dark-field - are amenable to tomo- 
graphic reconstruction. As in conventional X-ray tomography, the tomographic 
reconstruction of the logarithm of A(x,y) yields the spatial distribution of linear 
attenuation coefficient /r in the sample. Tomographic reconstruction of the DPC 
image [preceded by integration along x, or using an imaginary sign filter in the 
tomography routine (Pfeiffer et al., 2007a)] yields the distribution of X-ray refractive 
index S^. The DFI can be reconstructed in analogy to the attenuation image, and 
will yield the distribution of an effective scattering coefficient (Wang et al., 2009). 



7. 6.4. 2 Performance Characteristics and Applications 

XGI excels in terms of the high sensitivity and precision of the phase contrast it 
creates. In radiographic DPC projections obtained with XGI, a precision of down 
to lOnrad (in terms of refraction angle a) can be reached. XGI phase tomography 
data have a precision that can reach 0.5 mg cm^^ (mass density), corresponding 
to 2 X 10“^° in terms of X-ray refractive index, or 0.2 electrons nm“^ in electron 
density (Pfeiffer et al., 2007b; Herzen et al., 2009; Schulz et al., 2010; Zanette et al., 
2011a). The spatial resolution is limited by the diffraction on G1 and by the detector 
resolution. The limit imposed by diffraction on G1 scales with XD/p^. This means 
that an increase in sensitivity, by choosing a longer inter-grating distance D, will 
come at the cost of a degradation of spatial resolution. 

As with other phase-sensitive radiography methods, XGI performs best with 
highly-brilliant sources, such as those at SR facilities. However, compared to other 
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X-ray phase-contrast methods, the requirements of XGI on beam monochromatic- 
ity, beam parallelism, and detector resolution are relatively relaxed (Weitkamp 
et al., 2005a). In addition, if the transverse coherence of the beam delivered by the 
available X-ray source is not sufficient, an additional grating GO can be inserted 
between the source and G1 to solve the problem (Pfeiffer et al., 2006). In this way, 
XGI can be implemented with low-brilliance sources such as standard X-ray tubes. 

Quantitative interpretation of the dark-field contrast modality is the subject of 
intensive research (Yashiro et al., 2010; Bech et al., 2010; Lynch et al., 2011). 
Moreover, dark-field imaging can be extended, by rotating the sample around the 
optical axis, to yield not only the scattering strength in different regions of the 
specimen but also the degree of anisotropy and the preferential direction of the 
scattering structures (“directional dark-field imaging”; Jensen et al., 2010). Similar 
information can be retrieved if gratings structured in two dimensions are used 
instead of line gratings (Zanette et al., 2010). In moire mode (Momose et al., 2009) 
or using other advanced acquisition schemes (Zhu et al., 2010; Zanette et al., 2011b, 
2012), XGI can be made compatible with fast, real-time tomographic imaging 
(Momose et al., 2011). 

Grating interferometers are currently available to the user community at several 
facilities in Europe (McDonald et al., 2009; Weitkamp et al., 2010; Herzen et al., 2010) 
and Japan; the first tests were reported from North America (Richter et al., 2009). 
Test setups using standard X-ray sources have been described from various research 
laboratories (Donath et al., 2010; Jerjen et al., 2010), and a first implementation 
in a commercial laboratory X-ray p,GT scanner has recently been reported (Tapfer 
et al., 2011). Developments have been published from numerous research groups 
around the globe, and the availability of this method is rapidly increasing. Other 
grating-based techniques that, unlike XGI, do not use the interference effect should 
also be mentioned. They include coded-aperture phase contrast (Olivo et al., 2011) 
and Fourier X-ray scattering radiography (Wen et al., 2009). 

Among applications in the life sciences of this emerging method, we mention 
phase tomography of the rat brain (Pfeiffer et al., 2007b), human brain (Schulz et al., 
2010), and the human inner ear (Richter et al., 2009), and also in mammography 
(Stampanoni et al., 2011) and applications in food science (Jensen et al., 2011). In all 
these domains, the ultrahigh sensitivity of XGI makes it possible to visualize the 3D 
microstructure of soft tissue at unprecedented contrast. Dark-field imaging has been 
applied to dental structures in mammals (Jensen et al., 2010), where it reveals the 
presence and orientation of sub-resolution structures, such as tubuli in the dentin. 

7.7 

Applications (Post-treatment) 

7.7.1 

Segmentation - Visualization Methods 

The data provided by the previously described imaging technologies, such as MRI 
or GT scanning, generally consists of arrays of voxels, the 3D equivalent of pixels in 
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2D. We will focus on the case where each voxel holds a single scalar value, called 
intensity. In what follows, we describe methods that aim to help scientists explore 
these data by using a graphical computational technique, called scientific volume 
visualization. 

Scientific visualization is a computational technique at the crossroads between 
data analysis and processing, computer graphics, and human-computer interac- 
tion. It has two goals for scientists: 

• to provide insights into complex datasets to help understand and analyze data 
from a qualitative and quantitative point of view; 

• to create expressive and aesthetically pleasant illustrations to improve scientific 
communication. 

Several free and/or open-source software tools exist for processing and visualizing 
3D voxel arrays, most of them being based on a library called the Visualization 
Toolkit (VTK) for visualization and on a library called Insight Toolkit {ITK) 
for image segmentation and registration. All-purpose visualization tools are not 
limited to MRI and CT scan data, such as, for example, ParaView and Visit. 
Other tools are specialized, like MeVisLab, MedINRIA, MRI Studio, or Fiji (based 
on ImageJ). Commercial products providing image segmentation and 3D surface 
reconstruction tools also exist, such as Amira, Avizo, Mimics, and VGstudioMAX. 

In general, VTKs are based on the application of successive filters to the data. A 
survey concerning the computer science discipline “scientific visualization” as well 
as a description of various filters can be found in Hansen and Johnson (2004). 

Scientific volume visualization uses classification to assign labels to individual 
voxels. Classification attempts to compute sets of voxels, called classes, which 
correspond to distinct ROIs, such as anatomical structures. Most classification 
techniques are based on a prior segmentation of the voxel array (see below). Labeling 
is the final step of assigning a color and transparency to each class to compute visual 
illustrations. In what follows, we first describe the main visualization techniques 
and then some of the main classification techniques. 

7. 7. 1.1 Main Visualization Methods 

A voxel array of intensities, resulting for instance from (iCT, can be considered 
to be a stack of 2D images along one of the three coordinate axes, so that these 
2D images/slices can be displayed individually. Figure 7.15a depicts an example 
of slice visualization. Since displaying only 2D slices loses the third dimension, 
several elaborated visualization techniques have been developed. The objective is to 
restore a good perception of 3D structures. Among the most common techniques 
there are, on one hand, the direct volume rendering (DVR) techniques and, on 
the other hand, the simpler iso-surface display techniques. Both reconstruct a 3D 
object, which is then projected on the screen almost like a camera would take a 
photograph of a real-life 3D object. 

The iso-surface extraction technique consists of building a smooth 3D surface 
from a sub-set of selected voxels, that is, a sort of hull that bounds these voxels. The 
sub-set of voxels is either chosen using an iso-intensity value (i.e., a set of voxels 
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Figure 7.15 Different techniques for ren- positions, (d) iso-surface with cutting planes, 
dering a voxel array of intensities of Atlantic (e) maximum intensity projection, (f) addi- 
mudskipper, Periophthalmus barbarus. (a) tive volume rendering, (g) direct volume 
Displaying individual slices, (b,c) iso-surface rendering with semi-transparency, and (h) 
reconstruction for two different light source direct volume rendering with shading. 

matching a certain intensity value or a certain intensity interval) or as the result 
of more advanced classification (see below). The resulting surface is defined as a 
set of triangles and displayed using the graphics boards available in computers. 
To compute the illustration, scientists essentially have to define a projection (e.g., 
an observer position, as for a real camera) and some lighting conditions along 
with the surface refiectance properties. Shading is the procedure that computes a 
color according to the local surface properties (essentially the normal vector), the 
lighting condition, and the refiectance properties. Advanced shading techniques 
might further integrate shadows, light diffusion, or specific graphical styles (such 
as illustrative border enhancement). Color patterns that are pasted onto the surface 
may also be used to highlight local surface properties, such as mean or Gaussian 
curvatures. 

The result of the computation is a sort of “photograph” of the 3D surface, 
depending on the aforementioned shading technique. The chosen shading tech- 
nique influences the computation time, which can range from instantaneous to 
several minutes. When computation of the illustration is perceived as instanta- 
neous by the user (i.e., it is below about 300 ms), the visualization is said to be 
interactive. Interactivity is often a desired goal for both classification and illustration 
computation, as it offers users the possibility to browse various parameters in 
“real-time,” thus generally facilitating data exploration. The more complex the 
classification technique is, and the higher the number of visual effects integrated, 
the longer the computation of the illustration will take. Frequently, lighting and 
reflectance parameters are predefined or, at least, default values are proposed, so 
that the user first and foremost selects only a graphical “style” and the viewing 
direction, plus the zoom. It is also possible to define one or more cutting planes to 
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align the original X-ray (rCT data with the reconstructed surface. Figure 7.15b-d 
shows an example of interactive iso-surface visualization. 

The DVR technique is technically more complex. It computes an image without 
reconstructing an explicit 3D surface. It consists of sampling points along rays that 
pass through the 3D array of voxels. The samples are used to gather collections 
of ordered intensity values along each ray. These intensities are used to compute 
a corresponding color by applying a so-called compositing formula. Color and 
transparency values are assigned to intensities by using the transfer function. 

In its simplest form, the transfer function is a ID lookup table that links each 
intensity value to a color and transparency. The compositing formula, that is, 
the way the ordered colors and transparencies are combined, determines the 
resulting image. Some compositing schemes are order independent. It can be 
reduced to applying a maximum intensity operator, which keeps only the color of 
the sample with highest intensity value. This kind of DVR technique is called a 
maximum intensity projection (M/P). An example is shown in Figure 7.15e. Order 
independent compositing may also be based on the weighted sum of colors, with 
the weight being provided by the transparency. In this case, the technique is called 
additive volume rendering. An example is shown in Figure 7.15f Another common 
compositing formula, which exploits ordering, is inspired by the physics of light 
traversing the participating media: a semi-transparent corpus composed of small 
scale particles that emit, absorb, and scatter light, such as haze, fog, or clouds. 
Using this kind of compositing formula, interleaved structures appear more or 
less transparent. The further away, the less visible a structure will be due to light 
absorption. An example is shown in Figure 7.15g. For all of these compositing 
techniques, no lighting condition needs to be defined. Unfortunately, the human 
visual perception system is not used to easily identifying objects composed of 
semi-transparent materials. A solution to improve perception therefore consists of 
adding, as for surfaces, a shading procedure based on light sources and reflectance 
properties. Since no explicit surface is reconstructed and yet surface parameters 
are needed (such as the normal vector), the gradient of intensity is generally used 
as local surface descriptor. Figure 7.15h shows an example of DVR with shading. 

Many more compositing techniques have been proposed. Most of them attempt 
to guarantee that “important” internal structures remain “best” visible. Two recent 
examples are the studies by Chuang, Weiskopf, and Moller (2011) and Marchesin, 
Dischler, and Mongenet (2010). Other techniques attempt to improve transfer 
functions. In particular, some transfer functions do not only use intensity values 
but also local operators such as the gradient and/or the Laplacian. Instead of a ID 
lookup table, the transfer function becomes a 2D, 3D, or higher dimensioned table 
[see for instance Kniss, Kindlmann, and Hansen (2001) and Caban and Rheingans 
(2008)]. Some DVR methods further attempt to improve shading and shadowing 
without penalizing rendering performance (Solteszova et al., 2010) or propose to 
unify iso-surface visualization with DVR (Ament, Weiskopf, and Carr, 2010), the 
former being expressed as a particular case of the latter. 

One particularity of the DVR approach is that it allows one to take into account 
the entire set of voxels. All data are used to create the illustration. The counterpart 
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is that all data must be stored in the graphics processing unit, thus limiting the size 
of voxel arrays (arrays that are too large must be down sampled). Another difficulty 
with this technique is to define an appropriate transfer function. Most direct volume 
visualization tools propose some default functions, often predefined according to 
the scientific context (medical, geological, etc.), but there is no universal solution to 
automatically define a good transfer function. Scientists often have to experiment 
with their own functions, which can take a considerable amount of time. 

7.7.1 .2 Classification and Segmentation 

Defining transfer functions for DVR, as well as defining intensity intervals for iso- 
surface reconstruction, are both related to the notion of classification. Classification 
consists of associating to each voxel a given index called the class. By defining colors 
and transparencies using a transfer function, voxels become implicitly classified. 
Likewise, defining an intensity interval for iso-surface reconstruction consists of 
classifying the voxels. In fact, defining sets of intensity intervals is the simplest 
way of performing voxel classification. This classification technique is called 
thresholding. However, in most cases, threshold-based classifiers are not enough to 
discriminate and dissociate meaningful structures within datasets. The problem is 
that different anatomical components might have similar intensity responses. In 
addition, neighboring structures might cause blurred intensity values at boundaries, 
which is called the partial volume effect, that is, multiple tissues contrihute to a 
single voxel. All of this makes it difficult to differentiate structures using only voxel 
intensity. 

Some classifiers therefore use more information than just intensity, like the 
gradient, the Laplacian, or more generally, vectors representing responses to local 
filter banks, like the Gabor filter bank. These classifiers then apply a feature 
function to the computed information to perform the classification. A simple 
feature function is, for instance, the histogram. Often, local extremes, hills and 
valleys, in the histogram are used to define thresholds for partitioning intensities 
into intervals. 

Computational 3D segmentation is a technique aimed at automating or at least 
facilitating the delineation of connected components of interest in voxel datasets - 
purely manual voxel-by-voxel segmentation being an excessively tedious task. 
Formally, computational segmentation is defined by a procedure that results in 
the partitioning of an image I into n sets of distinct connected components C;. 
Segmentation should not be confused with the previously defined terminology of 
voxel classification. The distinction relates to the fact that classification produces 
sets of voxels that might be disconnected, such as for instance a collection of 
individual bones. Segmentation, on the other hand, provides sets of voxels that 
match connected components (each individual bone is separated). To some extent, 
segmentation can be considered as a particular case of classification. Most research 
concerning automatic segmentation is related to the field of medical imaging [see 
for instance the survey by Dzung et al, 2000]. 

Classification and segmentation are generally unified, which also explains why 
both are frequently confused. For instance, once individual components have 
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been segmented, classification might simply consist in grouping or connecting 
components that have been separated by the segmentation. This can be carried out 
either manually or automatically by analyzing, for example, shapes and locations 
of components. Classification based on segmentation might even be performed 
implicitly during the labeling step by assigning an identical color/identifier to 
separated components. It might also be completely skipped when the amount 
of classes matches the number of segmented components (i.e., each component 
represents exactly one class). Conversely, some segmentation techniques use 
classifiers, for instance based on feature functions. 

Different algorithmic methods for automatic segmentation can be listed (without 
being exhaustive): thresholding, region growing, classifiers, random field models, 
and atlas-based techniques. Since some of these techniques require manual inter- 
vention, they are called semi-automatic. For example, in its simplest form, the 
“region growing” algorithm requires manual selection of a seed position. Classi- 
fiers referred to as “supervised” use training data and perform a fe-nearest neighbor 
classification. Classifiers referred to as “unsupervised” avoid the use of training 
data. The so-called fe-means clustering algorithm is such an example. Atlas-guided 
approaches are based on a template, used as a reference/example to process new, 
yet similar, datasets. The core problem is then to register the dataset with the 
template. Because of the need for a template that is generally built by hand by 
expert scientists and because of the use of some knowledge related to the given 
scientific domain for performing the registration of the dataset with the template, 
atlas-based methods that are developed for one specific scientific domain cannot 
be easily transferred to another scientific domain. Despite decades of research, 
segmentation techniques are still not fully automated or precise. Some manual 
intervention (correction) is nearly always required. 

Once the voxels of the array have been classified, a color, which is a strong visual 
feature identifier, can be assigned to the class and displayed using either one of the 
two previously described visualization techniques. Figure 7.16 illustrates visual- 
ization using iso-surface rendering including shadowing effects. The classification 
is based on semi-automatic segmentation using region growing. Segmentation 
is followed by manual reconnection of separated components and grouping into 




Figure 7.16 Segmentation facilitates the delineation of regions of interest in voxel datasets, 
and allows visual classification of features by assigning them some individual color value. 
Volume rendering of (a) skin surface and (b) head and pectoral part of skeleton of the 
Atlantic mudskipper, Periophthalmus barbarus. Lens is black; scale bar=l mm. 
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classes belonging to the same anatomical structure (for instance, the eyes class, 
labeled with the color black, is composed of two separate components). 

7.7.2 

Numeric Simulation - Testing Hypotheses on Adaptive Evolution in Marine 
Vertebrates Using 3D Imaging Tools 

Three-dimensional volumetric models allow the testing of hypotheses related to the 
adaptive nature of phenotypic divergence in evolutionary biology. To do so, different 
modeling approaches, including both forward and inverse dynamic modeling, 
can be used [e.g., Aerts et al. (2003) and Van Wassenbergh et al. (2003)]. More 
specifically, multibody dynamics analysis and finite element analysis (PEA) provide 
excellent quantitative tools in this context [e.g., Curtis et al. (2008) and Moazen 
et al. (2009)]. For the past few decades engineers have successfully been using 
multibody dynamics and FEA to predict the behavior of engineering structures. 
As these simulations are driven by a demand for efficiency, their application to 
biological problems in general and marine biology more specifically makes sense. 
However, applying these native engineering techniques to questions related to the 
biomechanics of marine vertebrates is accompanied by specific challenges, as accu- 
rate geometry and the forces acting on the structure of interest are often difficult to 
obtain. Obtaining accurate material properties can be even more troublesome: in 
contrast to most engineering materials, like steel or concrete, bone and soft tissues 
typically show a nonlinear, anisotropic elastic behavior that greatly differs not only 
between tissue samples but also within one sample [e.g., the Young’s modulus and 
Poisson’s ratio of biological tissues may vary locally as the result of an epigenetic, 
mechanical control of bone remodeling; see, for example, Biewener, Swartz, and 
Bertram (1986), Carter, Van Der Meulen and Beaupre (1996), Currey (2003), and 
Chen et al. (2008)]. Even in the rare case that material properties are available, 
variation can be considerable (www.biomesh.org). This renders the interpretation 
of simulations with even the most advanced material modeling challenging. 
Luckily, obtaining quantitative values is rarely the primary goal of simulations 
in biomechanics. Usually, biologists are more interested in, for example, regions 
of high stress concentrations, although estimates of exact stresses may become 
crucial for studying skeletal failure under certain conditions (Soons et al., 2010). 
In addition, combining parts of a 3D model with different material properties is 
a challenge, but does confer relevance in terms of a proper understanding of how 
biological systems behave under natural loading conditions (Porro et al., 2011). 
Changes in the applied material model will typically only have a limited effect on 
the location of high stress concentrations. As such, simulations can, for example, 
predict where bone fracture is likely to occur under certain loading conditions, but 
will usually fail in accurately predicting under which loading conditions. 

Pew studies so far have focused on marine organisms, with studies exploring 
structural adaptation in the skull in burrowing eels (Herrel et al., 2011) or suction 
feeding in seahorses (Leysen et al., 2011) being an exception. Another study has 
focused on the caudal system in seahorses, where the tail has evolved into an 
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Figure 7.17 (a, b) Bending of the seahorse caudal skeletal structure simulated using multi- 

body dynamics. 

articulated, prehensile structure within the family of Syngnathidae (Hale, 1996; 
Van Cauter et al., 2010, Figure 7.17). Despite its high level of flexibility, the 
seahorse tail still shows a relatively high resistance against radial forces (which 
likely makes it an effective protective armor against predators). This combination 
of radial stiffness and ventral flexibility can only be achieved by the articulation of 
a chain of uniformly shaped skeletal segments. Each segment consists of a central 
vertebra, surrounded by four dermal plates. In on-going research, all of these 
skeletal elements are being modeled as rigid bodies, connected by spring elements 
that represent the joints. The effect of muscle contraction (hypaxial muscles and 
median ventral muscles) can be simulated and several parameters can quickly be 
adapted so that their effect on the kinematics can be quantified. In this way, it 
can be determined whether these parameters are optimized by evolution for tail 
prehensibility of a rigid system. 

Multibody simulations of the seahorse caudal structure can uncover the secrets of 
how radial strength and stiffness can effectively be combined with extreme bending 
flexibility - knowledge that could be a source of inspiration for breakthroughs in 
engineering fields that focus on high radial stiffness (like stent design). Further 
applications of 3 D models of biological systems can be found in translating the quan- 
titative data within the virtual environment into physical elements using different 
techniques of stereolithography and rapid prototyping [e.g.. Chapman, Andersen, 
and Wilcox (2003) and Nielson, Kaehr, and Shear (2009)]. This not only provides a 
means to generate models that can be used for display, but also allows hypothesis 
testing on adaptive evolution as well as designing innovative models inspired from 
nature, with possible applications in industry or even the medical world [e.g., 
Simaan, Taylor, and Flint (2004), Blevins and Lauder (2011), and Fish (2011)]. 



7.8 

Conclusion 

The oceans cover more than 70% of the Earth’s surface and almost 99% of the 
planet’s living space. However, despite 250 years of taxonomic classification, 91% 
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of the estimated 2.2 million of marine eukaryotic species still await to be described 
and classified (Mora et al., 2011). Moreover, the rate of data accumulation on 
marine organisms and the increase in the phylogenetic resolution of marine taxa 
are both slower than for terrestrial taxa (Thomson and Shaffer, 2010). Knowing 
this, and considering that the world’s oceans are undergoing profound changes 
due to anthropogenic climate changes (Jackson, 2008; Hoegh-Guldberg and Bruno, 
2010), there is an urgent need to conduct large-scale taxonomic analyses of marine 
environments. The best way to reach this objective is to develop data collection 
strategies adapted to the study of marine organisms. We believe that the large panel 
of tomographic methods presented here may be a valuable tool for this purpose. 

MRI constitutes a very convenient non-invasive and non-destructive in vivo 
method to access soft tissues information. In addition to conventional aMRI, fMRI, 
diffusion magnetic resonance imaging (dMRI), and MEMRI provide powerful tools 
to gain insight into biological processes in vivo (particularly in the nervous system). 
However, the necessity to use hydrated specimens prevents the use of alcohol- 
preserved specimens (most collection specimens) without prior rehydration. This 
and the long acquisition times required for high-resolution scans (up to tens of 
hours) compared to X-ray p,CT could constitute the main drawbacks of this method. 

Conventional X-ray p,CT is based on absorption and thus is not so well adapted 
to the imaging of animal soft-tissue at high spatial resolution, although it produces 
excellent results with bony or shell-bearing organisms because of the absorption 
difference between the bone (or shell) and the soft tissues. Staining agents can 
enhance the contrast between different tissues. X-ray PCI methods (free space 
propagation, ABI, and grating interferometry) offer contrast levels up to 1000- 
times higher than those obtained by absorption methods. This, combined with 
the high spatial resolution that can be reached by these methods, makes X-ray 
PCI a very appropriate technique for the study of the countless, small, soft-bodied, 
marine organisms. The disadvantage of these methods is that most of them have 
X-ray beam requirements that are only met at synchrotron-based setups. However, 
laboratory-based setups providing X-ray phase contrast at slightly inferior spatial 
resolution and sensitivity than synchrotron setups have been demonstrated and 
are under active further development. 

Regardless of the acquisition method, the data provided by imaging techniques 
(MRI, X-ray tomography) are, most of the time, first exploited by scientific volume 
visualization methods (DVR or iso-surface display, for example). Such visualization 
techniques allow the reconstruction of 3D objects that can be projected into 2D 
images on a screen and easily manipulated to obtain visual insight into complex 
datasets or to create scientific illustrations. These 3D volumetric models can then be 
used for numerical simulation methods like FEA or multibody dynamics analyses 
to test biomechanical hypotheses. 

Taken together, these methods offer the new ways needed to gather morphologic, 
anatomic, and physiologic data on marine organisms, from microscopic planktonic 
organisms to large marine mammals. 
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Imaging Marine Life with a Thin Light-Sheet 

jerkmie Capoulade, Emmanuel C. Reynaud, and Make Wachsmuth 



Abstract 

Light-sheet fluorescence microscopy (LSFM) is a non-invasive optical method 
for the observation of living specimens. Although this concept was estab- 
lished a century ago it is only during the last decade that instruments suitable 
for biological applications have been developed, thereby circumventing some 
of the limitations of established fluorescence imaging techniques such as 
confocal laser scanning microscopy. LSFM utilizes a sheet of laser light to 
illuminate only a thin slice of a fluorescently labeled sample. A wide-field 
fluorescence microscope, placed perpendicular to the light-sheet, serves to 
collect the fluorescence signal and image the observed region by means of a 
camera. This side-on illumination conflguration features several advantages, 
including intrinsic optical sectioning without the need of spatial filtering 
as employed in confocal microscopy, excellent signal-to-noise ratio, high 
temporal resolution, and drastically reduced overall photobleaching and pho- 
totoxicity inside living specimens. Moreover, the non-conventional geometry 
of LSFM opens up a completely new way of sample mounting, enabling con- 
venient multi-view image acquisition for 3D imaging by simple rotation of the 
sample within the medium-filled chamber. Although LSFM was developed 
originally for the observation of large organisms such as zebrafish embryos, 
this method can be adapted to a large range of samples from macroscopic 
specimens like corals or copepods to microscopic organisms like tintinnids. 



8.1 

Introduction 

Modern light microscopy provides access to the structure, arrangement, and 
dynamic behavior of living specimens, covering a wide range of applications from 
imaging of single molecules through cells and tissues to 3D imaging of complete 
organisms. As a non-invasive technique, optical imaging enables the study of 
samples while they are maintained under conditions as close as possible to their 
normal physiological context or the natural habitat. 
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Optical imaging exploits contrast, i.e., the spatial variation of the specimen’s 
optical properties such as absorption, reflection, scattering, fluorescence, and 
nonlinear effects. In particular the emergence of lasers and LEDs (light-emitting 
diodes) as intense, stable, and convenient light sources has considerably advanced 
the use of fluorescent light as microscopy contrast. Moreover, for living sample 
observations, light sources emitting in the visible and near-infrared spectral range 
are especially useful because these wavelengths are not harmful at the intensity 
levels commonly used for microscopy, even for long-term observation (Prasad, 
2003). This chapter will focus on the use of fluorescence as a microscopy contrast. 

In conventional fluorescence microscopes the sample is illuminated and 
observed through the same objective lens, that is, along the same optical axis, 
referred to as epi-fluorescence microscopy. It provides quite high lateral resolution 
but features limited axial resolution and considerable background signal that 
limits the image quality. Optical sectioning strategies like confocal microscopy 
address these issues by suppressing the detection of out-of-focus fluorescence. 
Nevertheless, the out-of-focus regions remain illuminated, leading to a loss of 
information during image acquisition. 

Light-sheet fluorescence microscopy (LSFM) (Huisken et al, 2004) has been 
established recently to overcome these limitations comprehensively by illuminating 
only a thin section of the specimen perpendicular to a wide-field microscope used 
for fluorescence detection. It improves significantly the signal-to-noise ratio by 
reducing the out-of-focus fluorescence excitation, while the phototoxicity and 
photobleaching are drastically reduced as less energy is deposited in the sample. 
Moreover, this method allows free rotation of the specimen within a medium-filled 
chamber for convenient multi-view 3D imaging. This is also closer to physiological 
conditions, especially when considering marine samples. Early observations of 
aquatic samples with a light-sheet microscope include marine bacteria (Fuchs et al, 
2002), zebrafish embryos (Huisken etal, 2004), and more recently copepods (Boistel 
et al, 2011) or even coral polyps, dinoflagellates, and tintinnids. As the sample can 
be held in an agarose rod or hooked, it can be maintained over days and allows 
long-term non-invasive studies such as copepod digestion (Karakoylii et al, 2009). 

In this chapter, we introduce LSFM in comparison to conventional optical 
sectioning and epi-fluorescence microscopy and propose it as a tool that is highly 
suited for the imaging of marine organisms. 



8.2 

Fluorescence Microscopy Methods 
8 . 2.1 

Fluorescence 

In optical imaging, the spatial variation of the specimen’s optical properties such 
as absorption, reflection, scattering, fluorescence, or nonlinear response is used to 
generate an image contrast. Transmission microscopy techniques based on phase. 
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dark-field, or differential interference contrast exploit the optical properties of the 
entire biological matter. Fluorescence contrast, however, relies on the capability 
of particular molecules, so-called fluorophores, to emit fluorescence photons in a 
certain wavelength range after the absorption of excitation light in a blue-shifted 
wavelength range (Lakowicz, 1999). By specifically attaching these fluorophores to 
molecules or structures of interest, their distribution, and dynamic behavior can be 
studied selectively with very high sensitivity. In addition, some intrinsic molecules 
of the specimen may also exhibit fluorescence, referred to as auto-fluorescence. 
Although this could interfere undesirably with specifically introduced fluorophores, 
it could also constitute a useful signal, overcoming in some cases the need to label 
living samples with specific fluorescent molecules. 

The energy of a molecule in an excited state is not inevitably transformed into a 
fluorescence photon but can also transfer the fluorophore irreversibly into a non- 
fluorescent state, a process called photobleaching that often involves the generation 
of highly reactive and thus phototoxic free radicals. In addition to phototoxicity, 
photobleaching depletes the pool of fluorescent molecules, therefore reducing the 
signal that can be recorded from the sample. 

The emergence of new probes, especially the genetically encoded fluorescent 
proteins (Miyawaki, 2011), and the high specificity of fluorescent tagging along with 
the dynamic technological development of lasers, optics, cameras, and computer 
power over the last 20 years has rendered fluorescence microscopy an essential and 
powerful tool in biology. 

8 . 2.2 

Epi-Fluorescence Wide-Field Microscopy 

The most widely used application of fluorescence imaging is epi-fluorescence 
wide-field microscopy where the sample is illuminated and observed along the 
same optical axis and through the same objective lens (Figure 8.1). LEDs, lasers, 
or mercury lamps serve as light sources for illumination, from which the desired 
spectral ranges are selected using filters or monochromators. The collected fluo- 
rescence photons, separated from the illumination using dichroic mirrors, may 
be further distributed and selected spectrally using filters and dichroic mirrors as 
well, and are subsequently collected with an imaging device such as CCD (charge- 
coupled device), EM-CCD (electron-multiplying charge-coupled device), or sCMOS 
(scientific grade complementary metal oxide semiconductor) cameras. 

Usually, the specimen is mounted in a medium-containing dish or chamber with 
appropriate optical properties and fixed on a horizontal stage that allows for its 
lateral translation in a manual or motorized way. The sample is observed either from 
the top or from the bottom using a conventional upright or inverted microscope 
stand, respectively. To acquire microscope images of a sample in 3D, spatial 
series of 2D sections at consecutive planes must be recorded by moving either the 
objective lens or the sample along the optical axis. Modern cameras allow up to 
100 frames per second to be recorded so that even a time series of 3D image stacks 
in several channels can be acquired to study the temporal behavior of a 3D object. 
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Figure 8.1 Concepts of epi- and light- 
sheet fluorescence microscopy. In the epi- 
fluorescence configuration, the same objec- 
tive lens is used for both the illumination of 
the sample and the detection of the fluores- 
cence (left). In the light-sheet configuration, 



a first objective (not shown here) is used 
to illuminate the sample from the side with 
a sheet of laser light, whereas the fluores- 
cence is collected with a second objective 
lens placed perpendicularly to the light-sheet 
(right). 



While providing such a good time resolution, a major drawback of epi- 
fluorescence wide-fleld microscopy is its non-isotropic spatial resolution: the lateral 
resolution is mostly limited by the diffraction of light whereas the resolution along 
the optical axis is substantially worse due to the corresponding axial symmetry 
of the optical path. Moreover, a significant and blurred contribution to the signal 
stems from sample areas outside of the focal plane and is referred to as out- 
of-focus fluorescence. This effect becomes even more prominent when moving 
the focal plane deeper into a thicker sample where both the illumination and 
the fluorescence light is increasingly scattered by cells, tissue, and other biological 
structures. Therefore, optical sectioning strategies have been developed to suppress 
the unwanted out-of-focus signal (Conchello and Lichtman, 2005; Fischer et al., 
2011; Haustein and Schwille, 2007; Stephens and Allan, 2003), as described briefly 
in the following paragraphs. 
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8.2.3 

Confocal Laser Scanning Fluorescence Microscopy 

To improve epi-fluorescence microscopy, the confocal laser scanning microscope 
(CLSM) has been devised (Pawley, 2006). Instead of illuminating and observing 
the complete 2D field of view at a time, the epi-fluorescence approach has been 
modified in such a way that only a single diffraction-limited region of the sample is 
illuminated with a focused laser beam. By scanning the beam in a raster-like fashion 
across the complete field of view, simultaneously collecting the fluorescence from 
the in-focus region and rejecting out-of-focus fluorescence light with a pinhole, 
excellent optical sectioning is achieved because, effectively, only fluorescence 
from a small focal volume confined in three dimensions reaches the detectors. 
However, the spatial resolution remains non-isotropic, although it is somewhat 
better than for epi-fluorescence wide-field microscopy. While the configuration 
of filters and dichroic mirrors is substantially the same as for epi-fluorescence 
wide-field microscopy, here point detectors like photomultiplier tubes, avalanche 
photodiodes, or hybrids of these technologies are used. 

One recurrent limitation of the CLSM comes from the sequential recording 
scheme of single-point microscopy techniques that slows down the acquisition 
speed remarkably, and thus limits the time resolution for 2D or 3D imaging. 

In addition, out-of-focus fluorescence light continues to be excited and is only 
rejected when it reaches the pinhole. Therefore, photobleaching and resulting 
phototoxic effects may still occur in a large volume along the whole optical axis. 

The latter drawback can be partially overcome by using two-photon laser scanning 
microscopy where the excitation is confined in 3D to a diffraction-limited volume. 
However, two-photon excitation can be quite inefficient and may yield unexpected 
photo-chemical and -physical effects. 

8.2.4 

Other Approaches For Optical Sectioning 

A solution to the speed limitation of point scanning microscopy is to parallelize the 
acquisition while maintaining the optical sectioning capability. The point scanner 
can be replaced by the typical core component of a spinning disk fluorescence 
microscope, a rotating disk carrying sufficiently spaced microlenses, which are 
illuminated with a laser beam and arranged in such a way that with one rotation 
of the disk the illuminated spots cover a 2D field of view exhaustively yet non- 
redundantly. Behind a second disk with confocally arranged pinholes, a camera 
catches the light during a full rotation, yielding a complete 2D image section of the 
sample. One of the main limitations of such a method is the light scattering that 
increases within increasing penetration into a thick sample, leading to spatial cross- 
talk between the numerous spots such that the optical sectioning and background 
suppression deteriorates substantially toward the quality of epi-fluorescence wide- 
field microscopy. 
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Structured illumination (and both CLSM and spinning disk fluorescence micro- 
scope can be considered as such) yet without structured detection allows to remove 
computationally the remaining out-of-focus and background signal post-acquisition 
by extracting the modulated contribution to the image and averaging over different 
complementary illumination patterns. Moreover, with appropriately selected pat- 
tern properties, the axial and the lateral resolution can be improved up to twofold. 
However, for every resulting 2D image, several acquisitions must be performed, 
which limits the time resolution and increases the deposited illumination energy 
and thus the probability of photobleaching. Being epi-illumination techniques, 
both spinning disk and structured illumination fluorescence microscopy are prone 
to out-of-focus illumination and photobleaching. 

Total internal reflection fluorescence microscopy (TIRFM) is based on an epi- 
fluorescence wide-field microscope but provides optical sectioning and avoids 
out-of-focus fluorescence excitation through axially restricted illumination. This 
results from the remaining evanescent laser light at a cover-glass-water (or - 
medium) interface when it is illuminated above the critical angle for total internal 
reflection. Fluorescence is only excited in the vicinity of the cover-glass surface and 
detected in the same way as for epi-fluorescence wide-field microscopy. TIRFM, 
therefore, allows fast imaging and provides a high signal-to-noise ratio. However, 
it cannot be used for 3D imaging of large samples, as it is limited to the areas close 
enough to the cover-glass surface (typically cell membranes). 

This list of optical sectioning techniques is far from complete and is covered more 
exhaustively elsewhere (Conchello and Lichtman, 2005; Mertz, 2011). Being mostly 
derived from epi-fluorescence wide-field microscopy, they all have their pros and 
cons, but are most notably either limited in spatial or temporal resolution. Light- 
sheet microscopy, conceived more than a century ago (Siedentopf and Zsigmondy, 
1902), has only recently been identified as a very suitable approach to address the 
above-mentioned restrictions in fluorescence microscopy for optical sectioning, 
fluorescence background suppression, time resolution, and photobleaching in a 
very balanced way (Huisken and Stainier, 2009). 



8.3 

Light-Sheet Fluorescence Microscopy 

8.3.1 

Concept 

The rationale of LSFM is to combine the advantage of optical sectioning and 
suppression of out-of-focus fluorescence excitation with the convenience, flexibility, 
and speed of wide-field fluorescence detection by uncoupling the optical paths for 
illumination and observation of the specimen. 

Unlike most fluorescence microscopy techniques, which are based on epi- 
illumination and detection, an LSFM is composed of two independent optical 
units, the illumination unit and the detection unit, orthogonally arranged around 
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a medium-filled chamber containing the specimen (Figure 8.1). The illumination 
unit generates a light-sheet, which excites a thin section of a fiuorescently tagged 
specimen and which coincides exactly with the focal plane of the detection unit. 

The fluorescence of the sample is thus only excited in and emitted from a thin slice 
sitting in the focus of the detection objective lens, providing the intrinsic optical 
sectioning capabilities of an LSFM. Moreover, the illumination of the entire field 
of view of the detection unit at a time allows the acquisition of a 2D fluorescence 
image directly with a camera in a wide-field detection configuration without the 
need for scanning. Therefore, it speeds up substantially the acquisition of images. 

Of course, this also simplifies and accelerates the recording of 3D stacks of a 
specimen: only a single scan of the sample or the light-sheet along the optical 
detection axis is required. 

8.3.2 

Implementations and Instruments 



So far, most LSFM instruments used are custom-designed optical setups. The most 
basic configuration is the selective plane illumination microscope (SPIM) (Greger, 
Swoger, and Stelzer, 2007). It is composed of the three main modules mounted 
horizontally on an optical table: the illumination unit, detection unit, and sample 
chamber (Figure 8.2). 




Figure 8.2 Schematic drawing of a selective plane illumination microscope (SPIM). The 
main optical components required for the construction of this instrument are depicted. The 
optional optical paths for two-color imaging are drawn in lighter colors. 
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8. 3. 2.1 Illumination Unit 

The wavelengths of the laser sources should be chosen to match the spectral 
properties of the fluorescent molecules used for labeling the sample. It is best 
practice to combine the beams from several laser sources into a single beam using 
dichroic beam splitters and mirrors in order to be as flexible as possible regarding 
the labeling of the sample. This is also a prerequisite for multicolor imaging. The 
most useful wavelengths are 488 nm (or 470, 491 nm) for the excitation of green 
fluorescent proteins like EGFP (enhanced green fluorescent protein) and 561 nm 
(or 568, 594 nm) for red fluorescent proteins like mCherry. An acousto-optical 
tunable Alter (AOTF) placed in the optical path of the combined laser beam allows 
selection of the desired wavelengths, adjustment of their power, and switching of 
the excitation light on and off during image acquisition. The laser beam is then 
expanded with a telescope to All completely the aperture of the successive optical 
elements. 

A strikingly simple way to generate the light-sheet is to use a cylindrical lens. It 
focuses the beam anamorphically in the direction collinear to the optical detection 
axis and leaves it unaffected in the other direction. The major drawbacks of such 
simple optical components are firstly optical aberrations that spoil the quality 
of the light-sheet, and thus the optical sectioning and background suppression; 
in addition, their numerical aperture (NA) is rather limited, leading to relatively 
thick light-sheets and limiting the resulting spatial resolution along the detection 
axis. It is therefore advisable to couple the cylindrical lens with an objective lens. 
Because of mechanical fitting issues between the illumination and the detection 
objective, the use of a long-working distance water-dipping objective lenses that 
can be inserted directly into the medium is important. This will also make sample 
handling easier. 

The focus of the light-sheet is placed in the area of interest within the sample, 
which lies at the same time in the focal plane of the detection objective lens. 

8. 3. 2. 2 Detection Unit 

The detection unit, arranged at 90° with respect to the light-sheet plane, is a simple 
wide-field fluorescence microscope. A high NA water-dipping objective lens is used 
to collect the fluorescence light that stems from the illuminated plane. An emission 
filter rejects the remaining laser light that may be scattered or refracted by the 
sample, and lets only the fluorescence photons pass to the camera. The image is 
focused onto the camera by means of the tube lens. Different camera technologies 
can be used. Because of their very high quantum efficiency of more than 90%, 
back-thinned/back- illuminated FM-CCD cameras are a good choice for most 
LSFM applications. The more recent sCMOS technology is a promising alternative 
when the acquisition speed is an important factor. Indeed, sCMOS cameras allows 
to obtain 100 frames of 4 million pixels per second at a quantum efficiency of 
up to 70%. 

The detection unit can be easily extended to two-color imaging by spectrally 
separating two fluorescent species with a dichroic mirror placed behind the 
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detection objective lens, and by focusing the two signals either onto two different 
parts of the same camera chip or onto two different cameras. 

8. 3. 2. 3 Sample Chamber and Movement Unit 

The horizontal configuration of the SPIM setup requires the use of a sample 
chamber containing medium to keep the specimen healthy during the experiments 
(Figure 8.3). Holes are designed on two sides of the chamber to insert the water- 
dipping objective lenses into the medium, and glass windows can be placed on 
the two other vertical sides for convenient visualization of the sample in the 
chamber. Perfusion pumps allow the medium to be replaced (or replenished) 
and to administer media of different compositions or drugs. Heating and cooling 
of chamber and medium can also be easily arranged. Before the observation, the 
sample is embedded in a small agarose droplet or rod that permits it to be positioned 
in front of the objective lens (for the mounting techniques see below). The agarose 
droplet or rod is fixed on a motorized rotation stage, which itself is mounted on a 
3D-motorized stage, the first one enabling rotation of the sample perpendicular to 
the optical axis of detection for multi-view imaging and the second one enabling 
the observer to precisely position and scan the sample across the light-sheet. 



Laser 

excitation 




Fiuorescence 



Specimen embedded 
in agarose or hooked 

Sampie chamber , 
fiiied with medium 



Fiuid stream for 
themaiization of the medium 



Figure 8.3 The sample chamber: The spec- 
imen is embedded in an agarose cylinder 
in order to hold it in the focal plane of the 
detection objective lens. A rotatable motor 
mounted on a 3D-motorized stage allows 
precise placing and scanning of the sample 



across the light-sheet for image stack acqui- 
sition. Tubes allow the chamber to be con- 
nected to a perfusion pump so as to control 
the composition and temperature of the 
sample medium. 
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8.3.3 

Spatial Resolution 

The spatial resolution of the LSFM depends on the optical specificahons of both 
the illumination and the detection unit (Engelbrecht and Stelzer, 2006). The axial 
resolution of the LSFM is driven by the light-sheet thickness, which itself is 
determined by the NA of the illumination objective lens. The smaller the NA, 
the thicker the light-sheet is. For instance, an NA of 0.8 would result in a thin 
light-sheet and an axial resolution well below a micrometer, whereas an NA of 0.3 
would lead to a light-sheet ~2 pm thick. The lateral resolution is determined by the 
NA of the detection objective lens. As in conventional microscopy, the higher the 
NA, the better the lateral resolution is. Here, a resolution well below a micrometer 
is generally achieved, since in most of the cases the detection NA is higher than 0.8. 

8.3.4 

Sample Mounting 

As mentioned above, for typical LSFM configurations, the sample is mounted 
on a platform that provides a large free moving space without using any primary 
support such as a slide commonly used in conventional microscopy. While for many 
decades sample support has been mostly based on the flat and horizontal surface 
of slides and coverslips, such new ways of sample mounting expand our view by 
allowing at the same time precise translation and full rotation of the specimen as 
well as flexible control of the medium (temperature, gas, ions, water, chemicals) 
while following its temporal evolution (such as growth in embryonic development). 
Nevertheless, several recent LSFM implementations employed more conventional 
flat support (Capoulade et al., 2011; Dunsby, 2008; Holekamp, Turaga, and Holy, 
2008; Wu et al., 2011). As a common rule for all sample mounting strategies 
used for fluorescence imaging, it is crucial to minimize the impact on the optical 
properties of the microscope. 

The most obvious option to maintain the sample in the desired position is to use 
hooks, tweezers, or clips made of metal or plastic (Figure 8.4). This simple method 
features a high stability while moving the sample during image stack acquisition. 
However, only a few users have applied this technique for the observation of large 
samples such as brains (Klohs et al., 2008) or fish tails (Engelbrecht et al., 2007). 

The second and most common mounting technique to hold the sample is to 
embed it in a transparent hydrogel, for example, consisting of agarose. This is 
facilitated by the use of a syringe or capillary to create a rod-shaped hydrogel block 
that can be easily adapted on a rotating platform. The hydrogel block can be tailored 
to the size of the sample, molded as a cup or chamber to accommodate sensitive 
samples such as plants or moving embryos that don’t grow when spatially confined, 
or adapted for mounting several samples at once. However, one must be careful 
with the choice of the hydrogel regarding melting temperature, stiffness, optical 
properties, or contaminants as most of the products available on the market have 
not been designed for microscopy but rather for electrophoresis, spectroscopy, or 
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Figure 8.4 Light-sheet fluorescence microscopy (LSFM) sample mounting: In most cases, 
the sample is embedded in an agarose rod suspended from above, (a) and (e). Additional 
mounting strategies include "hooking,” (b) and (f), mounting in containers or chambers 
(agarose, FEP...), (c) and (g), or flat mounting, (d) and (h). 



growth support. This mounting method can be applied to hold samples ranging 
from entire adult organisms like mosquitoes (few millimeters) to single yeast cells 
(few micrometers) (Taxis et al., 2006). 

Small transparent containers can also be used for mounting samples. They are 
generally made of specific plastic Teflon foils featuring appropriate optical prop- 
erties for microscopy, such as polytetrafluoroethylene (PTFE) (Keller et al., 2008a) 
or fluorinated ethylene propylene (FEP), and can be filled with low concentrations 
of agarose or methylcellulose (Kaufmann et al., 2012). This type of container is 
very useful for sensitive samples such as growing cysts and tissues, in vitro micro- 
tubules growth assays (Keller et al., 2008a), and long-term time-lapse 3D imaging 
of embryonic development (Kaufmann et al., 2012). 

Nevertheless, more conventional mounting based on Petri dishes and other flat 
supports can be used for LSFM as well, especially for samples that do not require 
multi-view imaging like monolayer culture cells. This approach is mostly used with 
upright configurations and is discussed in more detail in the following subsection. 

8.3.5 

Upright LSFM: an Alternative Configuration with Simplified Sample Mounting 

The idea of putting the light-sheet microscope into an upright position was already 
used in the first instrument (Siedentopf and Zsigmondy, 1902) and implemented 
for fluorescence imaging a few years ago (Holekamp, Turaga, and Holy, 2008). In 
this setup, referred to as a objective-coupled planar illumination (OCPI) microscope. 
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Figure 8.5 Upright configuration of an LSFM. The illumination and detection objective 
lenses are directly dipped from the top into a medium-filled Petri dish containing the sam- 
ple. This enables the use of conventional sample mounting protocols developed for conven- 
tional fluorescence imaging. 



the optical fiber that delivers the light-sheet and the observation objective lens 
are dipped into the sample chamber from the top. As a major advantage, this 
configuration renders possible the use of classical Petri dishes as a sample 
chamber (Figure 8.5) (Capoulade et al., 2011; Wu et al., 2011). This avoids the need 
for more complicated sample mounting methods such as those described above. 
Most conventional mounting and manipulation protocols developed for imaging 
living specimens in Petri dishes are applicable; the Petri dish must be filled with 
medium, and the objective lenses are directly dipped into the medium from the top. 
In addition, this configuration does not suffer from optical aberrations induced, 
for example, by embedding the sample in agarose, especially when using a tightly 
focused and thin light-sheet. Although this configuration allows precise positioning 
of the specimen, the observation of its expansion or growth, the control of the 
medium regarding chemical composition and temperature, or the rotation of the 
sample are no longer possible or at least difficult to implement. 



8.4 

Advantages of LSFM 

As mentioned above, uncoupling the illumination unit from the detection unit 
allows to illuminate the specimen only in the relevant plane, which is the focal 
plane of the detection objective lens, thereby conferring LSFM with inherent optical 
sectioning capabilities. Therefore, since no out-of-focus fluorescence is generated, 
the LSFM provides high contrast images, even deep inside large objects. 
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Figure 8.6 Comparison between CLSM and LSFM: sequential versus simultaneous acqui- 
sition, and out-of-focus versus no out-of-focus illumination. In the LSFM, the overall photo- 
bleaching is lower whereas the efficiency of information collection is higher. 



One important and recurrent problem in fluorescence microscopy is photo- 
bleaching of fluorescent molecules. LSFM addresses this issue in a very efficient 
way (Figure 8.6): since only the focal plane of the detection objective lens is illu- 
minated, photobleaching of ffuorophores outside of this plane is avoided. Thus, 
the overall photobleaching and the resulting phototoxicity are greatly reduced. 
This is a clear advantage compared to CLSM where, due to its epi-fluorescence 
configuration, the optical sectioning is obtained by illuminating a large volume of 
the specimen and by rejecting with a pinhole a large portion of the fluorescence 
collected, to observe just a small region in focus of the objective lens. 

For the same reasons, the information collection is much more efficient with 
an LSFM than with a confocal microscope: a larger part of the fluorescent photons 
generated are collected and used efficiently for imaging, and not wasted by spatial 
filtering. 

As another major advantage of LSFM, the wide-field configuration of the detec- 
tion unit allows high-speed imaging. No scanning is required to acquire a 2D 
fluorescence image while optical sectioning is maintained. The speed of image 
acquisition is only limited by the capabilities of the camera used and by the bright- 
ness of the sample. Owing to recent advancements in camera technology, a rate of 
100 frames of 4 million pixels per second can be achieved. 
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When mounting the sample vertically it is straightforward to perform multi-view 
imaging by simply rotating the hook, the agarose rod, or the plastic chamber. 

LSFM is applicable to a large range of living samples, from single cells (a few 
micrometers) to entire adult organisms (few millimeters), by adapting the light- 
sheet thickness and length to the size of the specimen of interest and the spatial 
resolution desired. For single cells, a light-sheet thickness below a micrometer is 
recommended to achieve sub-cellular resolution, while a thickness of a few microns 
is often sufficient for 3D imaging of large specimens. 

Taken together, these features make the LSFM image acquisition system very 
fast, enable high-resolution imaging with high contrast deep inside large biological 
samples, and minimize photoinduced damage of living specimen. 



8.5 

Howto Improve LSFM Imaging 

Despite its advantages, especially for 3D fluorescence microscopy of large spec- 
imens, LSFM does have certain conceptual drawbacks. Long optical paths both 
for illumination and detection inside heterogeneously refracting, scattering, and 
absorbing biological samples might lead to shadowing effects and loss of inten- 
sity with increasing penetration depth. Not least due to the perpendicular 
illumination-observation geometry, these effects are clearly visible as stripes 
as well as loss of contrast, intensity, and resolution along the illumination direc- 
tion. To address these two issues, the multidirectional selective plane illumination 
microscope (mSPIM) (FLuisken and Stainier, 2007) has been devised. The mSPIM 
features an even light-sheet in the entire field-of-view by pivoting the light-sheet 
at an angle of ±10° around the optical axis of detection. This allows to illu- 
minate the sample from different directions and to remove the stripes in the 
resulting image. To overcome the loss of image intensity and contrast due to 
the single-side illumination configuration of the basic LSFM, the mSPIM has 
two facing illumination objective lenses that render it possible to illuminate the 
sample sequentially from two opposing directions. The two images are merged to 
a final image exhibiting a substantially more homogeneous contrast, intensity, and 
resolution. 

As another drawback, a typical LSFM provides anisotropic spatial resolution: 
usually, the axial resolution is lower than the lateral resolution, which poses an 
issue for 3D image reconstruction. A solution can be found in two steps. First, one 
has to acquire a multi-view image stack of the sample by rotating the agarose rod 
embedding the specimen. Second, the multi-view image stack is transformed post- 
acquisition into a 3D image data set with isotropic resolution (Swoger et al., 2007; 
Verveer et al., 2007). For this purpose, it is important to rotate the sample properly 
to enable a correct fusion of the multi-view image stack. To account correctly for 
inherent deviations from an ideal rotation, the sample can be embedded in agarose 
together with sparsely distributed fluorescent beads as fiducial markers (Preibisch 
et al., 2010). As part of the post-acquisition image processing, the positions of the 
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beads are tracked throughout the whole multi-view image series as an intrinsic 
coordinate system such that the multiple views can be aligned correctly. Of course, 
multi-view imaging limits substantially the temporal resolution when imaging 
larger specimens in 3D over time. As a partial remedy, multiple views can be 
acquired simultaneously not only by illuminating but also by observing the sample 
using two opposing light paths, or even by multiplexing illumination and detection 
light paths (Krzic et al., 2012; Tomer et al., 2012). 

Isotropic resolution can also be obtained by using two identical objective lenses 
or at least the same NA both for illumination and detection (Capoulade et al., 

2011; Planchon et al., 2011). In this configuration, no computational processing 
as described above is required, because the isotropic resolution is provided by the 
microscope hardware. 



8.6 

Different Ways to Generate a Light-Sheet 

The illumination path of an LSFM is flexible, so that different ways to generate a 
light-sheet have been developed to further improve its imaging capabilities. 

8 . 6.1 

Digital Scanned Light-Sheet Microscope 

In a SPIM, a static light-sheet is generated using a cylindrical lens either directly 
or through an objective lens. While this is simple and robust, it is often difficult 
to maintain constant illumination intensity throughout the desired field of view. 
In a digital scanned light-sheet microscope (DSLM) (Keller et al., 2008b), only a 
single line of the sample is illuminated at a time with a focused Gaussian beam 
(Figure 8.7). The light-sheet is generated dynamically by vertically scanning this 
line across the field of view during the exposure time of the camera. This confers 
to the DSLM several advantages over the SPIM, especially while observing larger 
samples. First of all, the intensity of the line remains constant while it is scanned to 
form the light-sheet, resulting in a more homogeneous illumination of the sample 
as compared to SPIM. Second, since the laser light is focused into a single line 
rather than spread out over an entire light-sheet, a DSLM can provide a higher 
illumination intensity for a given laser power. Third, instead of a cylindrical lens, 
only corrected optical elements (achromatic doublet, aspherical, or scan lenses) are 
used to create the light-sheet, which usually feature smaller optical aberrations in 
the illumination optical path, thus increasing the image quality. 

8 . 6.2 

Incoherent Structured Illumination 

The image quality can be further improved by combining the DSLM with incoher- 
ent structured illumination (Keller et al., 2010) that has previously been developed 
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Figure 8.7 The different ways of gener- 
ating a light-sheet: in the SPIM (selective 
plane illumination microscope) configuration 
a static light-sheet is generated by a cylin- 
drical lens, whereas in the DSLM (digital 



scanned light-sheet microscope) configu- 
ration the light-sheet is dynamically gener- 
ated by quickly scanning a focused Gaussian 
beam along the focal plane of the detection 
objective lens. 



for epi-fluorescence wide-field microscopy to improve the optical sectioning capa- 
bilities and the image contrast (Neil, Juskaitis, and Wilson, 1997). While the sample 
is scanned in the DSLM fashion, the laser intensity is modulated at different fre- 
quencies and different phases. Here, for a given spatial frequency, each section of 
the sample is imaged at three different phases (0°, 120°, and 240°). The compu- 
tational combination of the images obtained leads to the removal of background 
due to scattering and out-of-focus fluorescence from the in-focus signal, therefore 
increasing the image contrast. This method has also been applied on a static LSFM 
setup using a movable grid placed before the cylindrical lens to structure the static 
light-sheet (Breuninger, Greger, and Stelzer, 2007). 

8.6.3 

Two- Photon Excitation 



Two-photon excitation exploits the fact that fluorophores can be excited by the 
simultaneous absorption of two photons of approximately half the energy (or twice 
the wavelength) needed for its single-photon excitation. Because of the very low 
probability that a molecule absorbs two photons at a time, two-photon excitation 
depends on the square of the incident laser intensity. It occurs therefore just at the 
focus of the objective lens where the intensity is high enough, yielding intrinsic 
optical sectioning capabilities. Moreover, to match the absorption spectra of typical 
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fluorophores, near-infrared light is required, which shows significantly reduced 
light scattering in biological tissues and thus increased penetration depth of the 
incident laser light into thick heterogeneous specimens. 

Recently, two-photon excitation has been implemented in LSFM both for statically 
(Palero et al., 2010) and dynamically generated light-sheets (Truong et al., 2011). 

Both studies show that two-photon excitation gives a slightly better axial resolution 
and that it allows much deeper penetration into large samples as compared to 
single-photon excitation LSFM. 

8.6.4 

Bessel Beams 

The penetration depth into the sample can also be increased by using Bessel beams 
instead of conventional Gaussian beams to generate the scanned light-sheet. While 
Bessel beams are focused, they possess two beneficial properties: they do not diffract 
over a certain distance and they are self-reconstructing when encountering small 
obstacles, thus minimizing the deformation of the illumination beam in an optically 
heterogeneous environment. Bessel beams were first implemented on a DSLM 
using one-photon excitation (Fahrbach and Rohrbach, 2010). Despite their advan- 
tages mentioned above, one-photon Bessel beams thwart the conceptual advantage 
of LSFM because they feature pronounced side lobes and generate undesired out-of- 
focus illumination. To resolve this issue on the detection side, incoherent structured 
illumination together with image processing post-acquisition as described above 
(Planchon et al., 2011) or a spatial filter such as a slit in an intermediate image 
plane of the detection unit for side lobe suppression have been used (Fahrbach and 
Rohrbach, 2012). Alternatively, excitation side lobes can be suppressed on the illu- 
mination side by using two-photon excitation. Here as well, only the main lobe will 
contribute efficiently to fluorophore excitation since the illumination intensity in the 
side lobes remains below the threshold for two-photon excitation (Planchon et al., 
2011 ). 



8.7 

Application Examples 

LSFM applications in marine biology are discussed in more detail below. Here 
we give the reader an impression of some rather remarkable results obtained 
previously with LSFM microscopes. 

One of the major goals of LSFM development was the fast and efficient 3D 
imaging of large fluorescently tagged specimens like embryos that could be 
followed over time during their development with high temporal resolution. Using 
SPIM, Huisken et al. were able to study Drosophila melanogaster embryogenesis for 
more than 15 h at two different planes inside the specimen, demonstrating that 
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even in relatively opaque samples, high-resolution imaging is possible (Huisken 
et al., 2004). 

The development of DSLM enabled time-lapse 3D imaging of a zebrafish 
embryo over the first 24h of development (Keller et al., 2008b). The DSLM-specific 
improvements made it possible to observe cell division processes and to track the 
movement (position, speed, and direction) of each single nucleus in the embryo 
and allowed the authors to propose a model for mesoderm formation in zebrafish 
embryos. 

Extending the mSPIM concept to dual-color imaging, Huisken et al. were able to 
perform high-speed imaging of the beating heart in zebrafish embryos at a frame 
rate of up to 69 frames per second. By combining this technique with optogenetic 
experiments, where the heart beat rate was optically controlled, the emergence of 
heart function during zebrafish embryonic development could be observed and 
analyzed (Arrenberg et al., 2010; Huisken and Stainier, 2009). 

The imaging not only of larger specimens but also of subcellular structures 
and processes with high spatial and temporal resolution has been a goal of LSFM 
developments. Using high NA objective lenses for both illumination and detection 
allowed the creation of a thinner light-sheet as compared to previous LSFM 
implementations and the achievement of an isotropic resolution of only 300 nm. 
The use of a static light-sheet permitted quantitative study of the diffusion and 
binding of nuclear proteins with sub-millisecond time resolution (Capoulade et al., 
2011). The combination of a light-sheet generated by a scanned Bessel beam with 
two-photon excitation enabled 3D time-lapse imaging of chromosome dynamics 
during mitosis (Planchon et al., 2011). 



8.8 

LSFM Combined with Other Fluorescence Techniques 

LSFM has also been used as a basis for other established fluorescence microscopy 
techniques, normally based on epi-fluorescence wide-field imaging or CLSM, like 
super-resolution microscopy (Celia Zanacchi et al., 2011), fluorescence correlation 
spectroscopy (FCS) (Capoulade et al., 2011; Wohland et al., 2010), fluorescence 
life-time imaging microscopy (FLIM) (Greger et al., 2011), and single-molecule 
tracking (Ritter et al., 2010). All these techniques benefit from intrinsic optical 
sectioning deep inside the specimen and from wide-field detection using a camera 
to either fully parallelize the data acquisition over the entire field-of-view or to speed 
up the image acquisition in 3D. 



8.9 

Light-Sheet Fluorescence Microscopy for Marine Biology 

To fully benefit from the advantages of LSFM, the sample, the fluorescent labeling 
approach, and the mounting method must be carefully considered. The optical 
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properties of the sample determine both the penetration depth of the light-sheet 
and the transmission of the fluorescence toward the detection camera: the more 
transparent the specimen is the deeper the light-sheet can penetrate, and the higher 
is the fluorescence yield. However, if the sample is slightly opaque and diffracts 
or scatters the light-sheet, for example, due to lipid vesicles or dense collagen 
fiber arrays, the optical sectioning and the imaging quality will decrease along the 
illumination axis, whereas a poorly transparent sample will also affect detection of 
the fluorescence signal. Moreover, a fully opaque sample that blocks completely 
the penetration of the light-sheet, for example, due to insect cuticular structures or 
bone, will limit imaging to the sample surface. 

The optical properties and thus the imaging quality of the sample can be opti- 
mized by choosing a suitable model organism, for example, transparent fish like 
medaka, particular growth conditions, for example, no phenol red in the growth 
medium to avoid autofluorescence, or a clearing treatment for non-transparent 
specimens, for example, with BABB.'* Moreover, it is important to aim for homo- 
geneous labeling of the sample: many large molecules such as dyes and antibodies 
do not penetrate deeper than 100 pm into a tissue slice or embryo so that the 
staining will not be evenly distributed throughout the specimen. This could result 
in misleading observations because central areas of the sample may look dark and 
“empty.” 

As in any other microscopy technique, specimen preparation is crucial, as badly 
labeled and deformed specimens will not take full advantage of the microscope. 

This is especially important for LSFM, where the specimen handling is often 
performed in a very different way because of the need to hold the sample vertically 
in front of the detection objective within a medium-filled chamber. Because 
illumination and detection occur along different directions, LSFM provides the 
opportunity to revisit established sample preparation methods and to conceive 
completely new approaches. Ideally, they enable us to maintain the specimen 
much closer to its actual physiological appearance and under conditions much 
closer to its natural habitat. 

In summary, specimen preparation for LSFM experiments must be mechan- 
ically stable, the specimen must be well supported to avoid distortion due to 
movement during imaging, and it must support the physiological aspects of the 
specimen. 

The mounting system should be flexible enough to allow the specimen to develop 
and grow and should not change its mechanical properties during observation or 
dissolve in the buffer. It should feature minimal impact on the optical quality of 
the microscope. For example, the refractive index of the mounting medium should 
be as close as possible to the refractive index of the buffer filling the imaging 
chamber to avoid light refraction that may increase the light-sheet thickness and 
impair the detection resolution. Both optical and physiological aspects have to be 
brought together, especially in live imaging where biocompatibility has to be taken 

1) BABB (also known as Murray’s clear) is a 1:2 or 1:1 mixture of benzyl alcohol and benzyl 
benzoate. 
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Figure 8.8 Marine samples observed with 
a light-sheet fluorescence microscope, (a) 

A copepod (ISOjim) observed from eight 
different angles that can be combined to 
provide a full 3D view of the animal (autoflu- 
orescence). (b) A single angular view of a 
Thermocyclops consimilis after DioCg staining, 
showing a more evenly distributed fluores- 
cence signal and a detailed view of every 
appendage (250|im); intensity projection of 
a complete stack acquired along one angle 
(Boistel et oi., 2011). (c) Rhabdeiio spiralis, 



a tintinnid (120 |im); an enlarged view of 
the theca shows details of the pore distribu- 
tion and size, which is below 1 |im for each 
pore stained with DiOCg. (d) A whole mount 
of a fixed coral stem inside the LSFM sam- 
ple chamber, (e) A complete retracted polyp 
imaged from (d). Single zooxhantella can be 
counted, (f) Zooxhantella distribution {green 
and red dots) in the tip of a single tentacle 
from (e) processed using Imaris: red: low 
autofluorescence; green: high autofluores- 
cence. 



into consideration. Generally, the samples are maintained in their integrity in a 
liquid-filled chamber. 

Figure 8.8 displays some images of marine samples observed with an LSFM, 
showing that light-sheet microscopy method is indeed well adapted to marine 
biology. 



8.10 

Summary and Outlook 



Fluorescence microscopy is an essential tool for biology. By labeling proteins or 
structures of interest with fluorescent marker molecules, it is possible to assess 
quantitatively biological processes in vivo with sub-cellular resolution. During 
recent decades, the ever increasing demand of biologists for higher sensitivity. 
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higher spatial resolution, longer as well as finer temporal sampling, and multicolor 
imaging has challenged the most widely used and established fluorescence imaging 
techniques such as confocal microscopy. This has triggered and accelerated the 
development of new fluorescence microscopes. In particular, developmental biolo- 
gists need quantitative data acquired in vivo with high resolution deep inside large 
specimens such as embryos. Another typical application is to perform time-lapse 
3D imaging over long periods (e.g., for the observation of embryogenesis). 

LSFM has been devised to overcome some of the limitations of epi-fluorescence 
microscopy. First, by illuminating from the side only a thin section of the sample, 
LSFM yields high-resolution images with high contrast even deep inside large 
specimens. Second, the LSFM concept generically reduces photobleaching and 
phototoxicity. Therefore, as a non-invasive optical imaging method, LSFM is 
ideally suited for living specimens and allows 3D high-resolution and high- 
speed acquisition. LSFM can be adapted to various samples ranging from adult 
mosquitoes (few millimeters) to yeast cells (few micrometers). 

So far, LSFM has been mostly used for cell and developmental biology appli- 
cations. Flowever, the first observations on tintinnids, copepods, zebrafishes, and 
even entire coral stems have demonstrated the capabilities and also the convenience 
of LSFM to image marine life in 3D with high resolution and in an environment 
that is as similar as possible to the physiological conditions and natural habitat. 

We believe that with an increasing number of both custom-designed and commer- 
cial instruments becoming available, LSFM bears great potential as an important 
imaging tool for marine biology. 
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Ex-situ Macro Photography of Marine Life 

Mattias Ormestad, Aldine Amiel, and Eric Rottinger 



9.1 

Introduction 

One of the biggest challenges facing the new generation of scientists is making 
an increasingly complex and technical field more accessible and understandable to 
the general public in order to increase scientific literacy. Being scientifically literate 
helps in acknowledging science as a pool of information about the world around 
us, and helps making appropriate decisions for both our community and ourselves. 
While the scientific literature is largely reserved for those active in the scientific 
community, beautiful images and illustrations appeal to all and are a very powerful 
media through which to share scientific observations with the public, and to educate 
future generations. In these days of web communication and fast-food information 
where we are overwhelmed by images on a daily basis, quality scientific images 
representing animals, cells, or organelles rarely seen by laymen can grab one’s 
attention, raise questions, open eyes, and highlight important scientific issues. 

Historically, scientific illustration, a time-consuming and technically challenging 
technique, was used to create extraordinarily detailed drawings serving as a visual 
record to document and describe scientific observations of not only the specimen 
under study but also of various biological phenomena such as biodiversity, embry- 
onic development, and animal behavior (Dionysius Ehret, 1736; Haeckel, 1868, 
1904). Today, digital photography provides a modern and widely accessible alter- 
native to illustration that allows a greater proportion of the scientific community 
to quickly and accurately document natural life science. These days, the presence 
of photographs in all branches of science is taken for granted. However, it was not 
until the end of the 1890s that scientific photographs first began to be included in 
scientific journals, increasing progressively in numbers and significance, until they 
finally reached the front covers of journals only as recently as the 1970s (Wilder, 
2009). These days, the presence of photographs in all branches of science is taken 
for granted. 

All types of photography, from infrared, throughout the visual spectrum, and 
even into the ultraviolet and X-ray wavelengths, as well as derived digital techniques 
such as confocal imaging and tomographic techniques [e.g., magnetic resonance 
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imaging (MRI), optical projection tomography (OPT), selective plane illumination 
microscopy (SPIM), etc.], are currently extensively used to share complex scientific 
observations among the research community. These images also play an increas- 
ingly important role in reaching out to the general public and educating future 
generations. A good example are recent publications from the Census of Marine 
Life project (www.coml.org) in specialized scientific journals (McIntyre, 2010; Webb, 
Vanden-Berghe, and O’Dor, 2010). Images taken during scientific expeditions are 
also often used with great success by various general public media as a way of 
communicating the scientific message to a broad audience. 

Although sampling for immediate or subsequent detailed analysis (ecology, 
physiology, morphology, molecular analysis, etc.) is the main and most important 
aspect during scientific expeditions, it should not be forgotten that there are several 
other very important advantages of scientific photographic imaging: 

• High-quality images can store additional important information that can be 
retrieved even after the samples have been processed. 

• These images can be added and annotated in scientific image databases and 
therefore provide useful information to an even larger scientific community, for 
outreach and, importantly, for educational purposes. 

• Imaging projects on scientific vessels may be the only commonality that connects 
the various scientific studies carried out onboard (from oceanographic and 
chemical studies to plankton and benthic sea creatures) and they also create a 
simple and understandable platform between scientists and the wider public. 

• Images taken during scientific expeditions constitute a legacy for future genera- 
tions, similar to the old scientific drawings that still amaze us today. 

Technical improvements in diving techniques (e.g., SCUBA dives up to 700 ft) 
and underwater imaging equipment [e.g., remotely operated vehicles (ROVsj that 
can go even deeper (up to 15 000 ft) have opened new frontiers in marine science 
such that previously undescribed species can now be observed and recorded in 
their natural environment (Webb, Vanden-Berghe, and O’Dor, 2010; Haddock 
et al., 2005). Nevertheless, underwater marine photography imposes constraints 
that may impair image quality or altogether prohibit shooting (e.g., organism size, 
behavior, dive time, weather, and water conditions). Technological improvements 
and miniaturization of imaging equipment, however, have also made it easier than 
ever to bring advanced imaging techniques, such as portable photographic studios 
and microscopes, into the field for in vitro imaging under a controlled environment. 

In this chapter, we describe how to construct an optimized and portable photo- 
graphic studio for photographing marine organisms in the laboratory, in the field, 
or even onboard a research vessel where space can be very limited. We focus on 
the photographic theory as well as photographic equipment, including the choice 
of lenses, lighting system, and aquaria required to capture high-quality images and 
video of a diversity of interstitial, free-living, sessile, benthic, or pelagic marine 
organisms. 
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9.2 

The Exposure 

First, we will introduce some basic photography terms that will be used in the follow- 
ing sections. Although all exposure settings in the camera are interconnected - for 
any given exposure the shutter speed, aperture size, and light sensitivity (ISO) 
of the film or sensor are all dependent on each other -we will present these 
terms individually and later suggest workflows that fit the needs for ex-situ macro 
photography of marine biodiversity. 

The exposure, or the total amount of light that reaches the image sensor (or 
the film), is determined by the shutter speed in combination with the aperture 
of the lens. Conventionally, the exposure is measured in units of exposure value 
{EV, also called stops), representing a halving or doubling of the exposure. Multiple 
combinations of shutter speed and aperture can give the same exposure - however, 
shutter speed and aperture size also affects other characteristics such as movement 
and depth of field (DoF), so even if the exposure is the same with different settings 
the resulting images can be very different! Thus, at any given exposure, doubling 
the time and halving the aperture size, or vice versa, will result in the same amount 
of light entering the camera. For example, a shutter speed of 1/125 s and f/11 is 
the same as with 1/250 s and f/8, or 1/500 s and f/5.6. In the following sections we 
describe how changing these values affects the exposure and the image. 

9.2.1 

Shutter Speed 

The shutter speed, or exposure time, is the length of time that the camera’s shutter 
is open to allow light to reach the film or digital image sensor. In addition to its 
effect on exposure, the shutter speed influences the way movement is captured 
in the picture. Short exposure times (i.e., faster shutter speeds) can be used to 
“freeze” fast moving objects (Figure 9. Id) and long exposure times (i.e., slower 
shutter speeds) can create blurry images if the camera or the subject is moving 
(Figure 9.1h). 

A rough estimate of the slowest shutter speed that can be used without creating 
blurry images due to camera movement is 1 /focal length of the lens being used. 

For example, to handhold a camera mounted with a 100 mm lens you use a shutter 
speed slower than 1/100 s, and even this requires you to have a very steady hand. 
However, this rule usually does not apply for macro shots close to the minimum 
focus distance (the minimum focusing distance of a lens is the closest the lens 
can get to a subject while maintaining sharp focus), which often requires a much 
faster shutter speed than the 1/focal length ratio. Image stabilization (or vibration 
reduction) mechanisms integrated into either lens (e.g.. Canon IS, Nikon VR) or 
camera (e.g., Sony) can often allow the use of shutter speeds two to four stops 
slower than the exposure requires. If a shutter speed is too slow for handholding, 
a tripod should be used to eliminate loss of focus because of camera movement. If 
you need to freeze movement in poor light conditions the only options are opening 
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Aperture Sensor sensitivity Shutter speed Magnification 



Figure 9.1 Basics in photography: (a) Rela- 
tion between light sensitivity and image qual- 
ity. A low ISO value corresponds to a lower 
light sensitivity, which means less noise (or 
smaller grain size on film) and higher res- 
olution and image quality, (b) Aperture or 
fstop. A high f-stop value correlates with a 
small aperture size, (c) Schematic represen- 
tation of the depth of field (DoF). The line in 
front of the fish indicates the beginning, and 
the dashed line the end, of the DoF in rela- 
tion to the aperture value. A low value gives 
a narrow DoF, while a high number offers 
a DoF that spans the entire fish size, (d, h) 
Real images illustrating the DoF described in 
(c). At f/2.8 (d) the dorsal fin is already out 



of the DoF, while at f/32 (h) the caudal fin 
is perfectly sharp and included in the DoF. 

(e, i) Sensor sensitivity: a low ISO value 
causes nearly no noise (e), while a high ISO 
value causes high noise (i). (f, j) Shutter 
speed: A high shutter speed (f) “freezes” the 
object, while a low shutter speed (j) causes 
a blurry image. To maintain a similar back- 
ground color, the image in (f) was taken 
at f/22, ISO 200 with flashes, while (j) was 
taken at f/5.6, ISO 800 and without flash. 

(g, k) Magnification: 1 :8 means that the 
object in the picture is eight times smaller 
than the real object while 4:1 means that 
the object in the picture is four times magni- 
fied compared to the real object. 
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up the aperture (see next section), adjusting the light sensitivity of the sensor (see 
later section), and/or adding flashes or fixed light sources that will provide the extra 
light needed for a good exposure. 

9.2.2 
Aperture 

An aperture is a hole or an opening in the lens through which light travels 
before reaching the film or the image sensor. Higher aperture numbers (i.e., f-stop 
or f- number) correspond to smaller openings, lower aperture numbers to larger 
openings (Figure 9.1b). Each sequentially higher f-stop (or f-number) allows half 
the light to travel through its opening than the previous one. Therefore, the aperture 
determines how much light reaches the image sensor. For a given exposure time, 
the narrower the aperture (higher f-number), the darker the final image will be. 

Note that all lenses lose sharpness when either “closed down” or “opened up” 
too much — each lens always has an optimal aperture range where the image has 
the best sharpness from the center all the way out to the periphery. Adjusting the 
aperture size also affects the DoF, which corresponds to the distance range over 
which objects are acceptably sharp (Figure 9.1c). The DoF is explained in more 
detail in Section 9.2.4. 

9.2.3 

ISO Value 

In traditional film photography, ISO (or ASA) is the indication of how sensitive a 
film is to light. ISO values are indicated in numbers (100, 200, 400, 800, etc.). The 
higher the value the greater is the sensitivity of the film, and the bigger the grain in 
the final image. In contrast, the lower the number is the less sensitive the film is 
to light and the finer the grain (Figure 9.1a). In digital photography, the term ISO 
has been conserved and indicates in a similar manner the sensitivity of the image 
sensor. Therefore, the same “sensitivity” principles apply as in traditional film 
photography but the “digital grain,” more properly called noise, has a completely 
different character than the grain on film. The noise on a digital camera is caused 
by random electrical charges in the sensor, and by amplifying the signal (i.e., 
increasing the ISO) you also amplify these random fluctuations. The appearance 
of this noise not only depends on the ISO value but also on other factors such as 
exposure time, sensor type, pixel density, and even the ambient temperature. The 
digital noise consists of both variations in pixel brightness and color. Most cameras 
have noise reduction built in and can for long exposures also make a dark frame 
exposure, which is automatically subtracted from the final image to remove sensor 
noise. 

Low ISO settings are usually used in good light conditions and will give highly 
detailed images with clear colors (Figure 9.1e) while higher ISO settings are 
generally chosen in darker situations, but at the cost of increased image noise 
resulting in the loss of fine details (Figure 9.1i). The ISO setting can be used to 
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compensate for lack of aperture size when keeping the exposure time down in low 
light conditions. For example, if you are using a 200 mm lens and at ISO 100 you 
obtain a shutter speed of 1 /lOO s at f/2.8 (let us say this is the largest aperture size 
for this lens), you will probably have a bit of camera-shake blur unless you have a 
tripod or a really steady hand. Turn up the ISO to 200 and you have the equivalent 
exposure at 1/200 s, or 1/400 s at ISO400. 

Changing shutter speed, aperture, and ISO is performed in EV steps or parts 
thereof (usually 1/3 steps), and by understanding these basic principles you can vary 
and balance these three factors to create an unlimited variety of images. 

9.2.4 

Depth of Field (DoF) 

A given lens can precisely focus at only one distance (focal plane) and the decrease in 
sharpness is gradual on each side of the focused distance. Nevertheless, within the 
DoF, the unfocused regions are imperceptible under normal viewing conditions. 
In normal photography the DoF is not evenly distributed around the focal plane, 
rather 1/3 of the field is located in front and 2/3 behind the focal plane. In macro 
photography, however, the DoF is equally distributed, with 1/2 of it extending both 
in front of and behind the focal plane. This might be good to keep in mind when 
obtaining macro photographs, since focusing on the eyes might not always be the 
best option if you want the whole animal within the DoF. 

A more scientific explanation of DoF is that the aperture size determines the 
cone angle of a bundle of rays that come to a focus in the image plane (e.g., 
image sensor), basically determining how gradual the transition to unsharpness 
is from the focal plane. The camera aperture (Figure 9.1b), which is adjustable 
in increments or f-stops (e.g., f/2, f/11, f/22, etc.), determines how co-linear the 
admitted rays are and therefore plays an important role in the appearance of 
the final image. If an aperture is narrow (high value, big DoF), then only highly 
co-linear rays are admitted, resulting in a sharp focus at the image plane (Figure 
9.1j). If an aperture is wide (low value, narrow DoF), then un-collimated rays are 
admitted as well, resulting in an image that is sharp at the focal plane but unsharp 
in all other parts or the photograph (Figure 9. If). 

An important fact to bear in mind is that the perceived DoF is not only determined 
by the aperture value but also by the focal length of the lens and in the case of macro 
photography also by the subject magnification. For a given f-number, increasing 
the magnification by moving closer to the subject results in a decrease in DoF and 
vice versa. 

The progress of digital technology and image editing tools has provided additional 
means of controlling the extent of image sharpness. For example. Helicon Software 
(Helicon Soft Ltd.) allows the DoF to be extended (by merging images taken at 
several focal planes). Tools like this can be very useful when the imaging setup has 
limited control of the DoF, such as a digital camera mounted on a microscope with 
a fixed numerical aperture of the lens. 
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For most scientific images it may be desirable to adjust the setting for a large 
DoF in order to have the entire object sharp and to capture most of the information. 

9.2.5 

Camera Settings 

Most current semi- or professional digital cameras offer various exposure modes, 
namely, P, Av, Tv, and M (or P, S, A, and M respectively for Nikon cameras): 

P (Program) corresponds to an automatic mode. The camera selects aperture 
and shutter speed while trying to keep a reasonable DoF and exposure time. 

Av (Aperture value) is the aperture (or f-stop) priority mode. The user selects 
the aperture and the camera adjusts the shutter speed accordingly. 

Tv (Time value) is the shutter speed priority mode. The user selects the shutter 
speed and the camera will select the aperture value to obtain a correct 
exposure. 

M (Manual) is a pure manual mode. The user has to set the desired aperture 
value and the shutter speed. This mode provides the most flexibility and full 
control over the shot. 

Modern cameras have several ways of measuring the light falling onto the sensor, 
and depending on how the light is measured you will obtain different exposure 
settings. For our purposes, however, light metering is seldom used since we actually 
do not want to make the exposure based on the natural light of the scene. We are 
making the exposure based on the light from our flashes. Since this is a situation 
where the flashes usually are at a fixed position, the background does not change 
unless we want to, and as there is no ambient light affecting the exposure, the best 
way is to use the camera in manual mode making sure we get the same exposure 
every time. The aperture or flash intensity is only adjusted when the magnification 
is changed, that is, according to the distance to the subject. 

9.2.6 

What Is a Good Exposure? 

This is of course a very difficult question and will have many different answers 
depending on what the photographer wants to visualize. In the context of scientif- 
ically documenting the anatomy of an animal, we could say that a good exposure 
would be focused on technical details such as a DoF that covers the entire animal 
and at the same time getting the maximum amount of color information possible 
with the image sensor. A good exposure in other contexts can, of course, be very 
different. 

A good way to get an overview of the color information in an image is by looking 
at the histogram (either directly in the camera or in an image editing application 
on your computer). A histogram (Figure 9.2) is a graphical representation of the 
tonal distribution of an image, plotting the tonal variations on the horizontal axis 
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Figure 9.2 The histogram, (a) This panel 
shows the sharp cutoff in the histograms 
when under- or over-exposing an exposure. 

In the middle is the original exposure, show- 
ing a nice tonal distribution over the whole 
histogram and to the left is an exposure 
underexposed by two stops, showing a cutoff 
in the shadow tones. To the right is an over- 
exposed image (also by two stops), show- 
ing a sharp cutoff of the highlights in the 
histogram, and also areas of clipped high- 
lights where detail is irreversibly lost, (b) A 
hypothetical exponential curve showing ten 



stops of dynamic range of a camera sensor. 
Half of the total bit-depth of the sensor is 
used by the first f-stop, on a 12-bit sensor 
there are 4096 levels of each color, mean- 
ing the first f-stop is 2048 levels, the sec- 
ond 1024 levels, the third 512 levels, and 
so on. (c) After gamma correction of the 
raw data, the tonal range is transformed 
to obtain an equal distribution of levels 
from dark to bright, (d) Examples of images 
of a tonaria larva on a black and a white 
background. 



and the number of pixels for each tone on the vertical axis. The horizontal scale 
ranges from dark tones on the left to light tones on the right. 

Learning to interpret the histogram can be very useful when determining whether 
an exposure is good. Again, “good” can mean very different things, but in this case 
we will consider “good” to be equal to capturing the maximum amount of color 
information possible. For a better understanding of the histogram and how to use 
it we will give an example and introduce a few new terms. 

First we need to have a basic understanding of how the image sensor works. 
A digital sensor has a completely linear relationship between the number of 
photons and sensor output, which is totally different to how an image is exposed 
on traditional film where the highlights are gradually compressed until the film 
saturates. Imagine each pixel as a bucket that fills up with an electrical charge as 
light falls on it, and when a bucket becomes full it just flows over. When a bucket 
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flows over - or rather when a pixel is saturated, all data that “flows over” is lost, 
resulting in loss of detail since differences in luminance between the different 
pixels disappears. This shows up in the image as “clipped highlights,” pure white 
areas where detail is abruptly and irreversibly lost. In the histogram, this looks 
like the pixel count for the brightest values never hits zero and there is a sharp 
cutoff at the right end of the histogram (Figure 9.2a). On the other extreme are 
regions in the image with very low light that fails to excite the pixels - or fill the 
buckets - above the background noise level. Similarly to the highlights, this will 
show up in the histogram as a cutoff in the left part (Figure 9.2a). 

Dynamic range is the luminance range over which the sensor can register 
light and is determined by, for example, the pixel size. Think of this as the 
surface area of the buckets; wider buckets have a greater chance of collecting 
enough charges to get over the background noise in low light regions of the 
sensor before the buckets in the high light regions overflow - effectively leading 
to an extension of the range where variations in luminance can be detected. 
Dynamic range is measured as a logarithmic scale and the unit is f-stops; as 
we described earlier, each f-stop has half the light of the previous one. When 
considering the linearity of the sensor output, this has some interesting implications 
for signal-to-noise ratios, as discussed by Michael Reichmann on his website 
“The Luminous Landscape” {http://www.luminous-landscape.com/tutorials/expose- 
right.shtml). Modern digital SLR cameras usually have a dynamic range of at least 
10 f-stops, which is comparable to that of black and white film. If we combine 
this with the color depth, or bit depth (see Section 9.6.1 for further explanation 
of bit-depth), of the sensor we can obtain a numerical representation of how the 
image data is distributed across the dynamic range of the sensor. This shows that 
half of the image bit-depth is contained within the first stop (Figure 9.2b), and 
about 97% of the available raw levels are contained within the five brightest stops! 

Since the human eye, similar to photographic film, does not have this linear 
response to light -twice the amount of photons are not perceived as twice as 
bright - this data needs to be transformed using nonlinear encoding called gamma 
compression or gamma correction. This is done by the RAW converter and essentially 
expands the darker tones and compresses the brighter tones, resulting in an equal 
distribution of levels from dark to bright (Figure 9.2c). If you underexpose a 
photograph and afterwards want to recover detail in the shadows you will have very 
little data to work with, resulting in increased noise and the risk of posterization 
of gradients within the shadow tones (remember that 97% of the bit-depth of the 
sensor is in the five brightest stops). To maximize the usage of bit depth you should 
therefore “expose right,” meaning that you balance the histogram to the right, but 
very carefully so as to avoid saturation of pixels. If you get more of the available 
stops to describe the shadows, you obtain a better signal-to-noise ratio and more 
shadow details can be picked up in post-processing. 

When shooting photographs against white or black backgrounds you probably 
want the background either as white or as black as possible (Figure 9. 2d). Inter- 
preting these histograms might be a bit tricky because you cannot know just by 
looking at the histogram if clipping in the highlights or shadows is just on the 
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background or also on the animal. In this case, most cameras have a highlight- or 
shadow-clipping indicator, highlighting the parts of the image that are in danger 
of saturation or loss of signal. 



9.3 

Using Flashes 

9.3.1 

Controlling Exposure 

To be able to use the desired DoF, to avoid blurry images, and to sufficiently and 
evenly light the subject, macro photography often requires the use of additional 
light sources. Most commonly flashes are used since they provide a good way of 
precisely adjusting the amount of light and they have a consistent and well-defined 
color temperature. As we have seen in the previous section, there are basically two 
options to control the light exposure of the sensor, namely, by adjusting either the 
aperture or the exposure time. However, in flash photography we are introducing 
one or more variable light sources that have to be balanced together with the 
exposure settings on the camera. 

In photography, flash synchronization (also called flash sync) is defined as the 
firing of the flash coinciding with the opening of the shutter allowing the light 
to reach the image sensor. Usually, the flash sync time can be anything down 
to 1/250 s. Even faster sync times are often possible when using the proprietary 
flashes for your camera system, but they work in a different way and significantly 
reduce the power of the flash. 

There are many ways flashes can be used when carrying out an exposure. The 
most common way is to use it as a fill light to modify the ambient light of the scene. 
For the purpose of the macro photography described here, however, we wish to 
reduce the ambient light as much as possible, thereby making the light from the 
flashes the sole source of light, giving us total control over the exposure. In this 
case, when the light from the flashes is totally dominant, the shutter speed has 
less significance for the purpose of freezing motion since the exposure is almost 
completely determined by the flash itself The duration of the actual flash is so 
short it will freeze even fast movements as long as the ambient light is not playing 
a significant part in the exposure. Most hot shoe flash units can be used in an 
automatic mode, TTL (through the lens metering), or in manual mode that allows 
the user to manually control the intensity of the emitted flash. 

9.3.2 

Positioning and Shaping the Light 

To obtain a high quality macro photograph it is important to light the subject 
correctly. When using flashes, you can control not only the amount of light 
produced by the flash but also the shape or character of the light by adding different 
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modifications to the light path or by moving the light source. Usually, the direct 
light from the flash is very “hard” and produces sharp highlights and shadows. 

For scientific or purely descriptive purposes it is usually important to have an 
even lighting of the subject with no pronounced shadows obscuring potentially 
important anatomical features. 

For the purpose of getting even lighting with few shadows that gives a good 
overview of the animal, you want a large light surface that scatters the light. This 
can be accomplished by using a diffuser, either in the form of a softbox (a sort of 
“tent” placed around the flash with a white semi-transparent surface that will act 
as the effective light source) or a smaller variant (e.g., Omni-Bounce or the built in 
diffuser on the flash). This will give the light a less pronounced directionality and 
will produce “softer” light and shadows (Figure 9.3b,f). 

Sometimes you need to direct the light in a very specific way, for example, to 
prevent it from illuminating a black background, or to concentrate it around a 
small transparent animal. To do this, a so-called snoot can be used, and it can be 
as simple as putting a rolled up piece of paper like a tube around the light source 
(Figure 9.3c,g). 

Usually, two flashes are enough when lighting smaller animals that will fit in an 
aquarium; the setup is usually one flash on each side of the camera to minimize 




Figure 9.3 Using the right flash accessories, 
(a, e) The flash: the use of only one flash 
(a) causes the formation of a strong shadow 
on one side of the object, while the use 
of two synchronized flashes allows a cor- 
rect illumination of the photographed object 
(e). (b, f) The softbox: The use of a softbox 
helps the user to soften the light on your 
object and enhances the white of the back- 
ground. (c, g) The snoot: The use of snoots 



helps to focus the light on the object rather 
than on the background. The use of a soft- 
box is recommended for white backgrounds 
while the use of snoots is recommended for 
black backgrounds, (d, h) Filtered seawater: 
The white dots and lines in (d) are reflec- 
tions of particles from non-filtered seawater. 
In (h) filtered seawater was used and the 
particle reflections considerably reduced. 
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shadows. A third flash is convenient when photographing larger animals against a 
white background since it can be used to specifically light the background, making 
sure it stays completely white and eliminates shadows from the subject. 



9.4 

Equipment 

9.4.1 
Camera 

Macro-photography of marine biodiversity can be realized with all sorts of camera 
bodies; the only requirement is that there must be some way to control a set of 
external flashes. However, since this work is mainly carried out in close proximity to 
water and often in the field, a rugged and well-sealed body is highly recommended. 
In addition, a full frame sensor (24 x 36 mm) may be more important than speed 
(frames per second) since larger pixels on the sensor gather more light and generate 
less noise. Presently, we are using the Canon EOS 5D mark II and Canon EOS 7D 
(both of which allow HD video recording). 

9.4.2 
Lenses 

Macro or close-up photography is defined by the subject-to-film (or digital sensor) 
ratio, and not by the distance from the lens to the subject, as the name would 
suggest - although a relation exists. This ratio, also known as the magnification rate, 
indicates the subject’s actual size compared to the size representing the subject on 
the digital sensor. Eor example, when the size of the actual subject is equivalent 
to the size of its image represented on the digital sensor, the magnification rate 
is 1 ; 1 (also called life size). Therefore, a ratio of 1 :8 (Figure 9.1g) indicates that 
the size captured on digital sensor is one-eighth of the actual size, while a ratio of 
4 : 1 indicates a four-times magnification of the actual size (Eigure 9.1k). Although 
no official cutoff for macro photography exists, real close-up photography usually 
starts at a ratio of 1 : 1. 

The lens defines the maximum magnification that can be obtained. The mini- 
mum focusing distance of a lens is the closest the lens can get to a subject, while 
maintaining sharp focus. At exactly this distance the maximum magnification rate 
of the lens is obtained. Traditionally, macro lenses have a fixed focal length that 
usually ranges between 50 and 60 mm, 90 and 105 mm, and 180 and 200 mm, 
depending on the manufacturer. The 50-60 mm lenses have a short minimum 
focusing distance, while the 180-200 mm lenses have a long minimum focus 
distance, which might be useful when performing macro photography in the wild, 
allowing the photographer to get close-up photographs of animals without getting 
too close and risking scaring them away. The 90-105 mm lenses are widely used for 
their intermediate working distance. Eor the macro photography described in this 
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chapter, the 100 mm macro lens is our lens of selection, with a few exceptions made 
for very small (Canon EF 65 MP-E 1-5 x) or bigger animals (Canon EE 24-70 mm 
non macro lens). You can certainly practice close-up photography without a macro 
lens, for example, using extension tubes, but the maximum magnification will not 
be as great. 

9.4.3 

Flashes 

Most amateur level and semi-professional camera models have a so-called pop-up 
flash built directly into the camera, but flashes can also be mounted onto the lens 
(e.g., macro- or ring flashes arranged in a circle around the front part of the lens) 
or mounted on the camera via the hot-shoe connector on the top. The use of such 
flashes will create reflections on the front panel of the aquarium and therefore 
obscure the subject. Using a cord or a wireless flash trigger, the flashes can be 
positioned independently of the camera and therefore undesired reflections can be 
avoided. 

For flexibility and maximum control of the lighting conditions, and also to avoid 
reflections in glass surfaces, we will focus on off-camera flashes synchronized with 
the camera either wirelessly (radio signal, IR, or by the built in flash), with a sync 
cable or with a TTL cable. Many flashes have built-in IR controls, so several flashes 
can be wirelessly connected with each other. By using a sync- or TTL-cable to a 
flash set as the “master,” this unit can control one or several “slave” units. When 
using wireless triggers there is usually a transmitter unit that needs to be put on 
the hot-shoe connector on the camera, with the exception of some Nikon cameras 
(the Nikon CIS, creative lighting system) and the Canon 7D, which have a built in 
control for remote flash triggering. An IR transmitter can be used to trigger flashes 
with built in IR receivers or an alternative is to use a transmitter/receiver system 
using radio waves, which have a much longer working distance, are not sensitive 
to daylight conditions, and more importantly do not require a direct line of sight 
between transmitter and receiver (this method however, requires you to have one 
radio receiver per flash). Both IR and radio systems can perform wireless TTL 
metering; however, most wireless TTL systems are expensive and are definitely not 
necessary for the purposes described in this chapter. A big drawback of TTL cables 
is that there are limitations in cable length; and for some strange reason they are 
all spiral shaped like an old phone cord. This may impede camera movements or 
result in the flash or the whole stand dropping to the floor. Therefore, the most 
flexible way of triggering your flashes is by using wireless triggers. 

For lighting we prefer the flexibility of hot-shoe flashes, and our current setup 
consists of a Canon Speedlite 580EX II and a Canon Speedlite 430EX, remotely 
triggered by either a Canon ST-E2 Transmitter (infrared) or AlienBees CyberSync™ 
radio triggers or by the built-in trigger in the Canon 7D camera. Additional 
advantages of using these flashes for this type of work is that they are relatively 
small and battery operated with readily available alkaline batteries (or rechargeable 
batteries) and can easily be used when no other power source is available. 
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9.4.4 

Flash Accessories and Lighting Gear 

Depending on the available space, we mount the flashes either on tripods or 
on flexible arms attached to a support. We also use 38 x 38 cm softboxes (e.g., 
Lastolite™, especially made for hot-shoe flashes) that can be mounted on tripods 
and positioned around the tanks if enough space is available. When space is limited, 
we also use smaller softboxes (13 x 18 cm) or “snoots” (to direct the light from 
the flash) that can be mounted directly on the light-emitting part of the flash. The 
flashes are mounted on flexible arms (also containing a hot-shoe connector) that 
are screwed onto a supporting plate. We use two types of support plates, one large, 
which deflnes the workstation, and a second, smaller in size, which is used to lift up 
(using Styrofoam) the medium size and plankton tanks (Figures 9.4 and 9.5). For a 
space limiting way to handle the lighting setup we use up to four flexible arms (e.g., 
two for flashes and two for LED lights to help with focusing in low light conditions). 

9.4.5 

Aquarium Setup 



As marine organisms are quite diverse in their behavior as well as in their size, 
the use of a set of tanks in different sizes and proportions is recommended. To 
reproduce the true colors of the animal or the plant we portrait and to achieve 
maximum light transmission, we opted for front panels in ultra white glass 
(Starphire® or OptlWhite^*^, Figure 9.5f). After testing various tank formats and 
sizes we also opted for custom-made rimless aquariums that are more scratch 
resistant and cause less distortions than acrylic tanks. Table 9.1 gives the sizes of 
the tanks and their usage. For both plankton tanks, the side panels should be made 




Figure 9.4 Photography studio equipment: tanks to allow imaging from above on a 

Schematic representation of the equipment black background, (c) Various support plates 

used to set up an ex-situ macro-photography for the tanks. The large plate contains four 
studio, (a) Four flexible arms hold the threads to fix the flexible arms, (d) Set of 

flashes and LED lights in the required posi- five aquariums in various dimensions. See 
tion. (b) Two Styrofoam blocks support the Table 9.1 for tank sizes and suggested use 
plate for the small tank and the plankton for each aquarium. 
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Figure 9.5 Macro photography studio: (a, 
b) Set up using the bottom plate; two Sty- 
rofoam blocks lift the support plate and the 
small tank, allowing imaging from above. In 
(b) two flexible arms holding LED lights are 
mounted on the back of the bottom plate 
and the tank contains a white plastic back- 
ground. Note that the white background 
fades out the back panel of the tank, (c) 

Set up using the bottom plate, two Styro- 
foam blocks, the support plate containing 



the insert to hold the large plankton tank, 
and two flexible arms, (d) Set up using only 
the bottom plate to hold a large aquarium 
and two flexible arms mounted on the front 
of the plate, (e) Profile of a small tank show- 
ing the curvature of the black plastic sheet 
that serves as background, (f) Detailed view 
of the OptiWhite® front panel (top) in con- 
trast to the regular greenish glass side panel 
(to the left). 



Table 9.1 Tank sizes and usage. 



Tank size 

(L X W X H in cm) 


5 X 5 X 2.5 


lOx lOx 5 


17x17x13 


30x15x20 


50 X 22 X 30 


Usage 


Small macro- 


Big macro- 


Small sea 


Medium sea 


Large sea 




plankton 


plankton 


creatures (up 


creatures (up 


creatures (up 




(2-8 mm) 


(1-3 cm) 


to 8 cm) 


to 12 cm ) 


to 15 cm) 



These five sizes cover most of the small to medium sized animals we imaged. Usage for these tanks 
may vary according to the behavior and character of the animals to be photographed. 

of thick matte glass so it can act as an integrated diffuser, helping to spread the 
light throughout the whole tank. 

9.4.6 

Other Accessories 

To obtain a clear view of the subject, monochrome plastic backgrounds can be used 
as backgrounds both inside and outside of the aquarium depending on the situation. 




226 I 9 Ex-situ Macro Photography of Marine Life 

It is usually advisable to keep a certain distance between the background and the 
subject, meaning that when taking photographs from above it is recommended 
to put the background a few centimeters outside of the tank. Backgrounds used 
inside the tank should cover the back and the bottom and fit tightly between the 
side panels so that the animals cannot hide behind them (Figure 9.5d). 

For high magnification, macro lenses such as the Canon MP-E 65 mm lens are 
suitable: a tripod in combination with a micrometric focusing rail is very useful at 
the higher magnifications. 

To remove particles from the water or to remove droplets from the front panel of 
the tank other accessories such as pipettes (various sizes). Chamois leather, or soft 
paper towels are very useful. 



9.5 

Macro Photography of Marine Life 
9.5.1 

Setting up the Camera 

For macro photography of marine organisms from about 10 to 20 mm and bigger, 
using a digital camera with the two flashes is the best option. For the best possible 
image quality the camera should be set to take RAW images, which contain the raw 
image data from the sensor and gives you more flexibility during post-processing. 
Joint Photographic Experts Group (JPEG) images use a compression algorithm 
that actually removes data from the image and, in addition, this format contains 
less image data to begin with, making it harder to make adjustments during the 
post-processing stage. It is also good to set the color space to “Adobe RGB” since 
this contains more color information than the standard “sRGB.” As color handling 
can be complex we will not go into details here. 

It is preferable to use the camera un-mounted (hand-held, rather than on a 
tripod), which make it easier to move around and focus on a moving animal. Since 
the flash will “freeze” any movement, handholding the camera will not result in 
unsharp images. Flowever, if you are going to film the animal, especially at higher 
magnifications, it is a good idea to use a tripod to ensure that the camera is steady. 
For more details and information concerning camera settings see Section 9.2. 

The camera mode has to be set to manual (M), and since the shortest flash sync 
time usually is 1/250 s this is what the shutter time will be set to. Since most of 
the light comes from the flashes the camera’s white balance setting can be set to 
“Flash.” To obtain a good exposure, you need to balance the power of the flashes 
with the lens aperture size and a good practice is to first set the camera to manual 
(M) with a shutter time of 1 /250 s and an aperture of f/20. Take a test shot with the 
flashes at half power (refer to the flash manuals to adjust the flash settings), and 
if the image is overexposed you have to either stop down the aperture (how much 
you can close the aperture depends on what lens you are using) or lower the power 
of the flashes. Usually it is preferable to stop down the aperture since we usually 
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want a larger DoF. If the image still is too dark with the flashes at maximum you 
need to open up the aperture to let more light in. This will result in a shallower 
DoF, so an alternative is to change the light sensitivity of the camera sensor (the 
ISO value), but if the ISO value is too high the flnal image will contain more pixel 
noise. For a more detailed description of the ISO settings see Section 9.2. The 
exposure settings will be very different, depending on numerous factors such as 
subject magnification, distance of flashes to subject, the kind of softbox used, and 
so on; after a while, though, you will get a feeling for what settings to start with. 

Each shoot generally takes a few shots to adjust the settings to get a good exposure. 

Working on a moving boat is quite different from standing on dry land — some of 
this type of photography may be quite difficult even if you are not moving around 
on the open ocean. If the boat is moving a lot you have to be careful not get splashes 
of water on the camera equipment, and focusing and composing the image will be 
a lot harder when you, the water, and the animal are moving. Using the camera’s 
autofocus with only the center focus point activated is probably a good idea but, 
unless you have a state of the art autofocus system and good static lighting, manual 
focus is the only realistic option. 

9.5.2 

Positioning the Flashes 

To obtain a clean background it is advisable to use a sheet of solid colored 
plastic (preferably neutral colors such as black or white depending on the color 
or transparency of the animal) behind and/or under the subject. This gives a 
background free of distractions, helping to emphasize focus on the subject. 

Lighting small creatures is usually easier than lighting bigger objects, which is 
why we will deal with it first. 

9. 5. 2.1 Lighting Small Animals 

Very small animals, and especially fast swimming small animals, are easier to 
photograph from above in a small tank or dish with a low volume of water 
restricting the vertical movement to a minimum. To create maximum contrast for 
small and transparent animals, side lighting is the key. Side lighting is preferably 
done using the two flashes facing each other on each side of the tank. To avoid stray 
light from the flashes entering the camera lens (resulting in a low contrast image), 
you can use so-called “snoots.” A cheap and easy method of making a snoot is to 
take a piece of paper (preferably dark and/or thick enough so that the light does 
not shine through) and roll it around the side of the flash head, thereby creating a 
narrow path for the light. 

Small and/or transparent animals are usually better visualized on a dark back- 
ground. A good way to obtain good lighting on the animal and at the same time 
obtain a really black background when taking photographs from above is to put the 
tank/dish with the animal on a stage with a piece of black paper or plastic placed 
about 5-10 cm directly under it. If the flashes are set up level with the animal using 
snoots, the background will receive little or no light and at the same time be out 
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of focus, ensuring a dark and homogeneous background. It is very important that 
there are no particles in the water since all particles will shine like little bright spots 
with the use of flashes (Figure 9.3d, h). If the photographs are taken from above, 
make sure that the bottom plate of the aquarium is clean and scratch-free. 

9. 5. 2. 2 Lighting Bigger Animals 

Generally, you want to put one flash on each side of the camera, pointing roughly 
at a 45° angle toward the center of the tank. You may need to adjust the positions if 
reflections appear in the glass. Usually, the biggest problem is to get the animal in 
the right position - especially if it is swimming! What you want is to create a contrast 
between the animal and the background, so, for example, a transparent animal is 
usually best viewed on a black background, since reflections and refractions in the 
anatomy of the animal will easily be visible against the dark backdrop. It is not nec- 
essarily always true that a dark animal will be best on a white background. The contrast 
might sometimes be too high, so it is always good to try a few different setups. 

With a white background we want it to be overexposed and washed out to 
pronounce the effect, but when using a dark background you do not want too much 
of the light falling onto it. Make sure the animal is not too close to the background 
and try to adjust the flashes so as to avoid the main light falling onto the background 
directly behind the animal. You can try to move the light so you obtain a smaller 
angle against the background and shield off the light going backwards by putting 
up pieces of paper or cardboard as a screen. If the animal is not too big and is not 
moving around much you can try using the snoots described in the previous section. 

In addition, when setting up the background for photographing larger animals 
from the side it is usually easier to get a good shot if the background covers both the 
bottom and the backside of the tank. If you use a large sheet that is bent between 
the two surfaces you will get a smooth looking background (Figure 9.5e). 

9. 5. 2. 3 Water Quality 

Regardless of the size of the animal, the purity of the water is crucial to obtaining 
good images. All water needs to be Altered (the best results are obtained with at 
least 0.2 pm pore size) to avoid the particles being lit by the flashes and appearing 
as dots on the final image (Figure 9.3d,h). Anything in the light path between the 
subject and the lens is important, so if you are taking the pictures through the glass 
of the tank it is of course also important to remember to clean the glass surface 
before you start shooting! Clean with freshwater first and finish off with some 
alcohol to remove any grease stains. 



9.6 

After the Photographic Shoot 

Although the main goal is to use an imaging setup that avoids extensive post- 
processing, factors such as the water quality can interfere with the quality of the 
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final image. If the lighting and camera are set up correctly and the water is filtered, 
you should not need to do much post-processing of the images. 

9.6.1 

File Formats 

On most current digital camera models, two file formats (RAW and JPG) are 
available. The RAW format is the raw output from the image sensor and is specific 
to each brand and model while JPEG is a standardized compressed format. The 
JPEG standard uses lossy compression, meaning that the compression algorithm 
removes information about the image to make it smaller. This compression, 
however, is done in a way that takes the sensitivity of the human eye into account, 
mostly removing luminance (brightness) and chroma (color) data in areas and in 
the ranges where the eye is less sensitive to details. This type of compression is 
especially suitable for photographic images where there are usually subtle gradients 
in color and brightness, and can achieve very high compression ratios without any 
major compression artifacts. This is a great image format for saving bandwidth 
when publishing photographs on the web and saving space when storing your 
family photographs, but there are several disadvantages that make this format less 
ideal for high quality photographs that might go through image processing and/or 
be used for large prints. 

You may think that the most obvious reason is the compression, and while this 
of course is an important factor affecting the image quality there is another major 
technical difference between the two formats - and that is bit-depth. The JPEG 
standard allows 8 bits of data allocated to each color channel (red, green, and blue), 
meaning that there are 2* = 256 levels of luminosity per color channel (this is 
the single-channel nomenclature meaning that 8 bits per channel gives an image 
with a total color depth of 8 + 8 + 8 = 24 bits). Taking all three color-channels into 
account we end up having a total color depth of 2^^^ = 256^ = 16 777 216 different 
colors. As the human eye can distinguish about 10 million different colors (Judd 
and Wyszecki, 2005) this might seem to be more than enough, but it actually 
does not give much room for adjustments. The native bit-depth of the raw output 
from most camera sensors is usually 12 or 14 bits per channel, meaning that 
a 12-bit image can address 2^^ = 4096 levels of luminosity per color channel -a 
considerable difference from the 256 levels of an 8-bit image. Therefore, keep in 
mind that when shooting JPEG you trust the camera software to down-sample your 
image data from the native bit-depth to 256 levels in the JPEG standard, which is 
a reduction to only 1/16 of the levels in a 12-bit raw image. Eor these reasons it is 
recommended that all adjustments to the image are carried out at high bit-depth 
and down sampling to 8 bits per channel is only performed with the final image 
before it is printed or published on the web. 

Another big difference in addition to compression and color depth is image 
processing. When using JPEG in the camera you let the camera software do 
adjustments to, for example, contrast, color saturation, and sharpness. These 
settings can usually be adjusted, but since this is done indiscriminately to all 
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images there is still a fair chance these settings will be suboptimal for many of the 
images. When shooting RAW you get to adjust these parameters manually and this 
lets you optimize each image individually. 

The RAW workflow is obviously more time consuming than handling JPEG, but 
powerful tools such as Adobe Lightroom® or Apple Aperture make the effort of 
post-processing minimal while still giving you maximum flexibility. 

9.6.2 

Post-processing Workflow 

We use Adobe Lightroom and Photoshop® for managing the images and to carry 
out image processing, although in most cases Adobe Lightroom is sufficient (at 
least from version 2 and upward). Our standard workflow is described in the 
following section, but should only be considered as a starting point. Each image, 
and especially each sea creature, has particularities that need to be taken into 
account both when doing the exposure and during post-processing; 

1) Check white balance and correct if necessary. If the camera is set to auto white 
balance it usually works fine with few exceptions. Since you are only using 
flashes, you may also set the white balance camera setting to “flash.” 

2) Sometimes cropping and/or rotation of the image are required. 

3) Remove spots and debris, and so on, from the background. 

4) Correct the exposure if needed, but you should make sure you get it right in 
the camera! If you shoot RAW images you will have far more possibilities to 
make exposure changes since the amount of image data in the RAW image is 
considerably larger than when shooting, for example, JPEG. 

5) Carry out level and curve adjustments to enhance contrast. 

6) Add a little more color saturation to reflect the true colors (just be careful 
so as not oversaturate any colors; the “Vibrance” control in Lightroom 2 is 
better to use than “Saturation” since it tends to be less prone to oversaturate 
colors). This is usually only necessary when shooting RAW since when using, 
for example, JPEG this is done automatically in the camera. 

7) Apply sharpening (e.g., “Smart Sharpen” or “Unsharp mask” in Adobe 
Photoshop or similar features in Adobe Lightroom). This should always be the 
last step. 

To produce lower resolution images suitable for web publication, you must 
remember to carry out another sharpening after resizing. In addition, if the 
color space of the image is “Adobe RGB,” convert the image into “sRGB.” This 
is because many web browsers do not take image color space into account and 
computer screens can usually not display all colors in the “Adobe RGB” color-space, 
so the image might look a bit “flat” with unsaturated colors. Adobe Photoshop, 
for example, reads the image color space and converts it for the computer display 
profile, so that the extra colors provided by the color space are only important 
during editing where it provides more room for making adjustments. 
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9.6.3 

Selecting Images 

When carrying out the photo shoot, try to make as many variations as possible of 
each animal, for example, with different backgrounds and from different angles. If 
you have time during the shoot it is also worth experimenting a little so as to obtain 
more artistic images suitable for education, promotion, magazine covers, and so 
on. From a large selection of photographs you will have a better chance of getting 
the material you perhaps did not even know you needed! 

Here are just a few examples of different image styles (Figure 9.6): 




Figure 9.6 Sample images: (a) psychedelic 
wrasse [Octopus cyanea), (b) Hawaiian day 
octopus [Octopus cyanea), (c) undefined 
larvae, (d) black nudibranch [Dendrodoris 
nigra), (e) undetermined juvenile crab, (f) 
crustacean [Phronima) protecting its eggs 
inside a salp (tunicate), (g) close-up of the 
mouth region of the cushion sea star Cul- 
cita novoeguineae, (h) upside down jellyfish 
[Cossiopea andromeda), (i) juvenile green 
sea turtle [Chelonia mydas), (j) swimming 



flatworm [Pseudoceros ferrugineus), and 
(k, I) brittle star [Ophiura fragilis). Images 
(d, g) have been used to illustrate the cov- 
ers of EMBO Journal in 2009 (volume 28, 
number 11) and 2010 (volume 29, number 
20), respectively. Image (d) has been fea- 
tured in Burn Magazine. Images (c, f) have 
been featured in the TARA Magazine and (c) 
has been used for a dress design by French 
fashion designer Agnes B. 
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’ science (all in focus, characteristic behavior, etc.); 

• art (movement, selective focus, creative reflections); 

• education (cute, fun). 



9.7 

Animal Care 

We also place particular emphasis on the preservation oflive animals in appropriate 
conditions after their capture, as well as the proper handling of the animals, before, 
during, and after the photographic session. This includes release into their original 
habitats directly following a photographic session, if they are not to be subsequently 
used for further scientific examination. 

9 . 7.1 

Collecting 

Scientists need to take buckets with them on the Zodiac (or other collecting vessels) 
to keep the samples that will be imaged permanently in the water and avoid 
additional stress for these animals. This is particularly important when the animals 
are exposed to high temperatures and direct sunlight. The use of a portable aerator 
that allows the continuous oxygenation of the samples is highly recommended. 
Remove only samples that actually fit in the tanks being used, and choose small to 
medium size samples, which are easier to handle and light. 

Sampling days on scientific cruises are usually busy with a lot of different 
samples that need to be collected and processed. Nevertheless, if serious imaging 
on board is envisaged, a dedicated “imaging sampling” protocol and time set aside 
for doing the actual imaging and postprocessing is required. The main purpose 
of performing macro-photography out in the field is to get photographs of live 
material and the time needed for this has to be taken into account when designing 
the sampling protocols. 

9 . 7.2 

Handling and Imaging 

Collected live animals should be kept in running seawater as long as possible, for 
example, in polystyrene boxes. Never keep the animals exposed to direct sunlight. 
Running seawater is important for animal health, oxygenation, removal of particles, 
and mucus from collected animals and for polyp extrusion in the case of corals. 
When storing several animals in the same container, make sure that they are 
actually compatible. Some animals, like nudibranchs, release toxic mucus that 
could be lethal for other animals in the tank and, therefore, they should be kept 
isolated. 

If possible, gently rinse your samples inside the polystyrene box (and change the 
water in the box to remove the sediment and mucus) before depositing them inside 
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the imaging tanks. If the animal still releases mucus, this can be removed using a 
plastic Pasteur pipette. 

For corals, extruded polyps are often retracted when one transfers them from 
the storage tank to the imaging tank, and it often takes quite a while before the 
polyps extrude again (if they do so). To work effectively, the use of several imaging 
tanks in parallel is very convenient. By doing so, the samples can be deposited 
simultaneously in the individual tanks, and while one sample is going to be imaged 
the others have time to relax and extrude their polyps. 

To ensure the best possible image quality, and in many cases also the well being 
of the animal, it is important to have access to clean filtered seawater, preferably 
from a 0.2 pm filter. 

Some animals (e.g., squids and nautilus) are extremely sensitive to light, and too 
much stress can cause the death of the animal. If imaging such animals, reduce 
the flash intensity to the strict minimum, or use no flash but, instead, raise the 
ISO settings. 
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Automated Image Processing in Marine Biology 

Robert T. Kelly*, Maureen A. Williams*, and Emmanuel C. Reynaud" 



10.1 

Introduction 

Planktonic communities are vital to many of the earth’s processes, most notably 
biogeochemical nutrient cycling. Plankton accounts for over 50% of global carbon 
fixation each year and often plays crucial roles in many marine food webs (Giovan- 
noni and Vergin, 2012; Jardillier et al., 2005). Moreover, in contrast to terrestrial 
ecosystems, aquatic environments contain plankton as the main form of biomass 
(Stingl and Giovannoni, 2005). 

Planktonic communities are composed of phytoplankton, plankton that are 
autotrophic and can produce their own food source, bacterioplankton, free living 
bacteria that function within planktonic zones, and zooplankton, which are het- 
erotrophic. A thorough analysis of the planktonic community can reveal much 
about the community’s current status as well as its evolutionary history. It has 
been demonstrated that planktonic community composition can affect the optical 
properties of water and control nutrient dynamics in oceanic systems (Boyd and 
Newton, 1999; Stramski, Bricaud, and Morel, 2001). Gommunity structure itself is 
affected by pH, grazing pressure, climatic variation, and seasonal variation (Klug 
et al., 2000; Qin and Threlkeld, 1990; Sanders et al., 1989; Werne et al, 2000). 
The study of plankton is one of the few areas where majors discoveries are still 
to be made and baseline studies are still needed. Planktonic community study is 
now legally required with the development of new legislation focused on whole 
ecosystem study, such as the Marine Framework Strategy Directive (EG, 2008). 
The most recent developments in the field of planktonic community analysis have 
been in the field of semi-automated planktonic community analysis, a field that 
has grown recently due to advances in computer technology. 

Several methods have been developed for planktonic community analysis (Hal- 
legraeff, Anderson, and Gembella, 1995; Sournia, 1978). Planktonic community 
analysis began with basic light field microscopy techniques and taxonomic classi- 
fication. The first recorded planktonic community study was conducted in 1828 

* equal contribution 



Imaging Marine Life: Macrophotography and Microscopy Approaches for Marine Biology, First Edition. 

Edited by Emmanuel G. Reynaud. 

© 2014 Wiley-VCH Verlag GmbH & Co. KGaA. Published 2014 by Wiley-VCH Verlag GmbH & Co. KGaA. 



236 I 70 Automated Image Processing in Marine Biology 

by Dr J. Vaughan Thompson (Fraser, 1968). Victor Hensen, a physician, then 
developed the popular Hensen net following much thought and study on the quan- 
titation of planktonic analysis and coined the term “plankton" based on Homer’s 
Odyssey (Smetacek, 1999). In the late nineteenth century. Per Teodor Cleve began 
using silk nets to study the distribution of phytoplankton in the North Sea (Cleve, 
1899). By this point, interest in marine fish stocks encouraged many scientists 
to begin examining planktonic communities and soon Hans Lohmann developed 
centrifugation techniques for concentrating plankton, enabling the discovery of 
nanoplankton (Lohmann, 1911). The first attempts to determine worldwide pat- 
terns of planktonic species distributions occurred in the mid-1950s, with Smayda’s 
meta-analysis (1958). Hobbie’s development of fluorescence techniques for phyto- 
plankton identification in 1977 again revolutionized the field, allowing the analysis 
of this important component of the community (Hobbie, Daley, and Jasper, 1977). 
Silhouette photography was first used in 1979 as a way to avoid issues associated 
with sample preservation (Ortner et al., 1979). Cameras for analysis of planktonic 
communities have even been included on submersibles (Wiebe and Benfield, 
2009). The most recent developments have been in the field of semi-automated 
planktonic community analysis, a field that has grown recently due to advances in 
computer technology. 

The importance of community analysis has thus grown and it has become 
increasingly important that marine scientists, legislators, and the general public 
understand the processes involved in planktonic community analysis. This chapter 
will present the current options available for analysis and the steps used to 
analyze these communities. There will be a focus on the current status of semi- 
automated planktonic community imaging and the issues associated with this type 
of community analysis in particular. 



10.2 

Methods of Planktonic Community Analysis 

10 . 2.1 

General Methods 

Plantonic community analysis is a multi-step process, with the time required for 
analysis increasing with the complexity of analysis. The most basic method of 
screening is the manual method, in which a scientist looks down a microscope and 
can either capture an image for subsequent analysis or can directly identify what they 
see. This method requires the least cost. However, it is also the most time intensive 
and can be difficult due to the lack of availability of standard datasets. The manual 
identificiation method requires the use of high-quality microscopes and skilled tax- 
onomists. A more efficient method involves the use of an automated imaging plat- 
form, which images all available samples and produces a large volume of images that 
can then be reviewed by the scientist. Current options are based on semi-automated 
identification, where each individual is analyzed as a geometric particle and then 
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classified based on its geometric measurements. The ideal system would be a 
semi-automated classification system in which the automated imaging is com- 
bined with an automated classification system, with machine learning allowing the 
system to provide a filtering mechanism, separating classes from the sample based 
on pre-existing datasets. This will allow for the final stage of the process to be a 
simple case of the scientist reviewing the choices made by the system. 

10 . 2.2 

Semi-automated Options 

Newer techniques, such high-throughput and high-content screening, allow higher 
replicate numbers and faster results. Semi-automated techniques allow for error 
estimation, which has been shown to be equivalent to error estimates of many 
taxonomists, and are reproducible to a higher degree (Benfield et al., 2007; Culver- 
house et al., 2003). The current options for semi-automated planktonic community 
analysis include hybrid optical-net systems such as the camera-net system (Olney 
and Houde, 1993; Ortner, Hill, and Edgerton, 1981). More recently, stand-alone 
imaging systems have been developed, such as the video plankton recorder (VPR, 

Davis etal, 1992), ZooScan (Gorsky et al., 1992, 2010), in situ video profiler (Tiselius, 
1998), shadowed image particle profiling evaluation recorder (SIPPER, Samson 
et al., 2001; Remsen, Hopkins, and Samson, 2004), zooplankton visualization and 
imaging system (ZOOVIS, Benfield et al., 2003; LISA, C. P. Gallienne), and the 
FlowCAM (Sieracki, Sieracki, and Yentsch, 1998). 

All of the semi-automated planktonic community analysis options treat the 
individuals as particles, only examining geometric parameters such as the ones 
shown in Figure 10.1, which inherently reduces the information associated with 
each image. The morphology, taxonomy, and geometry are separated in geometric 
classification, reducing the potential for interesting and informative analysis. By 
classifying solely on geometry and size parameters, it is possible for two members 
of the same species of different size to be classified as different taxa. Separating 
geometric, morphological, and taxonomic information reduces the power of the 
analysis and further introduces bias into classification systems. 



/ ■ 50 urn 

(M 

(a) ^ 



Figure 10.1 Geometry of two planktonic organisms. The measured parameters of a 
dinoflagellate (a) and a diatom (b) are shown. The height of each is shown in red, with the 
width in blue, and the spherical diameter in yellow. Morphological differences, such as the 
flagella of the dinoflagellate, are ignored, leading to issues with classification. 





238 I 70 Automated Image Processing in Marine Biology 

10.3 

Semi-automated Planktonic Community Analysis 

10.3.1 

Work Flow of Analysis 

Semi-automatic analysis of plankton is based on an adaptable workflow from 
capture to classification (Figure 10.2). The general process involves community 
sampling and pretreatment, image capture, data importation, preprocessing of 
images and data in silica, development of a classifier, classification of geometric 
particles, and final analysis of results (Figure 10.2). 




Figure 10.2 Workflow of semi-automated planktonic imaging. The steps taken from image 
capture to final analysis are shown. 
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10.3.1.1 Sample Collection 

Plankton community analysis always begins with the selection of a sampling 
method, most of which are time and labor intensive and generally involve the use 
of plankton nets. Following sampling, plankton can be analyzed in two different 
ways, via direct live imaging or fixation. The amount of time between sampling 
and analysis often dictates whether live imaging is possible. It is generally more 
informative to study live plankton species, as there are no fixatives, no cell shrinking, 
and color is maintained. Time constraints make live imaging difficult and therefore 
fixation can allow for more long-term observation. Difficulties arise rapidly with 
fixation, as cells can shrink and swell and samples degrade over time, impacting 
on taxonomic analyses (Deuer, Lessard, and Satterberg, 2001). Furthermore loss of 
cells can introduce bias into the system as some species may be more susceptible 
than others to degradation, with some species experiencing cell loss of up to 70% 
(Leakey, Burkill, and Sleigh, 1994). 

10.3.1.2 Image Capture 

The ability to create a suitable large reference dataset of images for plankton 
screening is strongly affected by the imaging platform used. For plankton, there are 
three primary locations in which an image may be taken, in the laboratory setting, 
on a boat with a dedicated imaging system, or using an underwater camera system. 

Each system of capture has specific positive and negative aspects and method 
selection should be carefully considered relative to the goals of each study. 

An early example of a high content method of screening came with the develop- 
ment of the VPR (Davis et al., 1992). This system used the principles of forward 
light scattering to image the nearly transparent specimens. Several cameras are 
used in the imaging system to observe several different size scales at once. This 
method provides high-resolution imaging and does so without handling the species 
themselves, allowing imaging of delicate or gelatinous species with impunity. How- 
ever, this early method was limited in its ability to define plankton at the species 
level and therefore is unsuitable for the current need for qualitative high content 
screening (HCS). 

The downside to these underwater imaging platforms is that they are nor- 
mally expensive and highly specialized, meaning that they are unavailable to 
many researchers. Therefore laboratory based fluorescence microscopes and high- 
resolution bright-field microscopes are more commonly used in plankton imaging. 
Wide-field microscopes can be useful in plankton studies due to their high reso- 
lution. Wide field microscopes enables retention of information at much higher 
magnification on individual wells, whereas lower resolution systems, such as the 
binocular, are better at imaging entire plates. However, in classification analyses, 
the binocular fails to provide the required resolution, and is therefore unsuitable 
for studies requiring such detailed qualitative information. 

The use of these types of imaging systems highlights the need for automation 
analysis as it is not only the capturing of images that is important in classification 
but the subsequent preprocessing of the image, which will usually take up most of 
the sampling time. 
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Digital images can reveal a wide variety of important information about sus- 
pended particles, such as abundances, size spectra, and distribution of planktonic 
biomass (Sieracki et al., 2010). Different intensities of light are converted into 
discrete points known as pixels, which vary in values known as gray levels, which 
in turn give contrast to the image (Cardullo and Hinchcliffe, 2007; Sieracki et al., 
2010 ). 

Regardless of the technical quality or expense of the system, the production of an 
image is usually accompanied by the formation of noise in the image, or variations 
in these gray levels, usually caused by uneven illumination due to scattering of light 
that can be observed as blurred or darkened samples (Uddin et al., 2011). Noise 
is particularly prevalent in plankton analysis due to the large quantity of particle 
samples in the images, which allows for more noise to form due to the packed 
nature of these particles. Preprocessing of the image is therefore an important 
step as this seeks to correct transformations that either the optics or electronics 
impose on the image, distorting the necessary information (Wolf, Samarasekera, 
and Swedlow, 2007). 

10.3.1.3 Importation and Segmentation 

Analysis with semi-automated planktonic techniques requires significant image 
preprocessing, the general steps of which are shown in Figure 10.3. After the 
images are taken, they must be imported into the analysis software. Importation 
can alter the images, reducing their size and making them easier to analyze. 

Generally, filters are used to enable quick and simple analysis and improve the 
image quality. One of the principal methods of image preprocessing is the process 
of adding filters to the image to attempt to remove noise. Several filters exist and 
many specialize in a particular task. In the Image) platform, common examples 
are mean and median filters, and the Gaussian blur filter. In median filtering, the 
neighboring pixels are assessed by intensity and the median value becomes the 
new value for the central pixel. This allows much of the noise to be phased out with 
respect to the surrounding environment and is particularly adept at limiting the 
infiuence of pixels with extreme values. It also contains various levels of precision, 
allowing a large degree of specificity. 

However, irrespective of the filtering technique used, one must acknowledge 
that any preprocessing technique will have a detrimental effect on the quality of 
the image. For example, using the median filter can drastically affect the number 
of pixels present in the image (Senel, Peters, and Dawant, 2002). In essence, it is 
better to take a good image first than to require the use of filtering at all. This also 
means that the quality of the camera used can be lost when using these techniques. 
The huge resolutions of some of the more expensive models will essentially be 
wasted upon subsequent preprocessing of the image. 

In most cases, an image is imported and a threshold established to generate 
a binary mask. This removes the image of the particle from the background. 
An outline can then be generated for further analysis. These steps introduce a 
bias, as thresholding values can significantly alter the particle’s shape and later 
classification. Many software options will turn color images into grayscale images 
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Figure 10.3 Image processing overview. The common steps in creating a classifier are 
shown. Following capture, all steps are completed in analysis software. There are many 
options for analysis, though most follow these general principles. 

to reduce the overall size of the image, though this again introduces bias and 
provides a source of variation in analysis. The way images are imported into the 
software can thus introduce a bias and care should be taken throughout. 

Particles are identified in the raw image in a process known as segmentation. 
Segmentation can cause issues when individual plankton are not segmented out 
of raw images in one piece (Figure 10.4). This issue is avoided when the image 
capture device captures only single images of each individual, though this increases 
the time associated with analysis. This is the step where the greatest bias can be 
introduced, as some particles are segmented out multiple times and some particles 
are not segmented out at all. 

10.3.1.4 Classifier Generation 

At this point, a taxonomist applies several filters to enable the software to recognize 
specific individuals that are representative of each taxon being studied. Representa- 
tive plankton must be identified and sorted by an expert taxonomist into their taxa, 
a process that requires time, expertise, and money. Expert identification and sorting 
introduces variation into the analysis at two points, reducing the ability of scientists 
to repeat the analysis and gain identical results. The images chosen to compose the 
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Figure 10.4 Common issues encountered 
in semi-automated planktonic community 
imaging. These FlowCAM® images, taken 
following processing in Zoolmage and Image 
J, demonstrate the wide variety of issues 
encountered in analysis, (a) Multiple angles 
of the same taxon are commonly captured 
due to the use of flowing fluid, as seen with 
the copepod in (b), where individuals cap- 
tured in aggregates are typically left: unclas- 
sified, as seen with the encircled copepod 



and dinoflagellates. (c) Plankton movement, 
as well as movement of the capturing appa- 
ratus, can lead to blurry images, shown in 
the dinoflagellates. (d) Segmentation can 
capture multiple planktons in one image, 
treating them as one particle for analysis, 
or (g) can capture only part of an individual, 
(e, f) Issues with debris and bubbles occur 
commonly, as seen in the detritus in (e) and 
the bubble in (f). 
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Table 10.1 Common algorithms used in classifier development. 



Algorithm 


Source 


^Learning vector quantization (LVQ) 


Kohonen (1987) and Tang et al. (1998) 


*Random forest (RF) 


Breiman (2001) 


5-Nearest neighbor {5-NN) 


Peters, Hothorn, and Lausen (2002) 


C4.5 


Quinlan (1993) 


NBC 


Ye, Chang, and Hsieh (2011) 


DINN 


Falchi et al. (2007) 


DKLL 


Demers et al. (1992) 


*Linear discriminant analysis (IDA) 


Hastie, Tibshirani, and Buja (1994) 


*K-nearest neighbor (KNN) 


Peters, Hothorn, and Lausen (2002) 


*Tree 


Breiman (2001) 


'"Neural network (NN) 


Simpson et al. (1992) 



Several algorithms have been developed for machine learning and classifier development. Most 
software packages will include multiple algorithm options when creating a training set. Asterisks 
indicate algorithm options in the FlowCAM software. 



training set affect the final results, as the angle from which the individual is viewed, 
the clarity of the image, and the background debris in the image can all affect the 
machine’s interpretation of how the taxon is identified, though these issues are 
less likely to impact a taxonomist. Each taxonomist also introduces variation, as 
his or her classification impacts the machine’s learning and the specific training of 
the taxonomist can bias classification. The use of several taxonomists helps avoid 
many of the issues associated with machine learning, though the matter should be 
considered in any analysis. 

The classifiers of most software utilize algorithms with the images of the plankton 
and the parameters to form a working model. Table 10.1 shows several algorithms 
commonly used. 

10.3.1.5 Classification 

Once the machine has been taught to recognize characteristics of the parameters 
associated with each particle class being studied, classification of unknown plankton 
images can occur. Particle characterization is automated and is the step that saves 
the most time for analyzers, in addition to eliminating the issues of taxonomist 
fatigue. However, there is much bias introduced by algorithm selection, image 
selection for the training set, individual taxonomists used, and the ability of the 
taxonomist to use the software. 

There are many software options available for analysis, though most are inex- 
tricably linked to the capture method used. Each piece of software has positive 
aspects, though many are not inherently user friendly and often require interac- 
tions with developers and some code troubleshooting. Often, solutions to common 
problems are addressed in online forums and blogs, where supplementary codes 
can be shared (Alvarez, Lopez-Urrutia, and Nogueira, 2012). There are many highly 
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taxonomically specialized software options, including DiCANN (Dinoflagellate Cat- 
egorization by Artificial Neural Network) (Toth and Culverhouse, 1999). DiCANN 
can analyze and classify dinoflagellates with neural network algorithms with up to 
72% accuracy. This taxon-specific software is accurate and used globally, though 
its high specificity is limiting. More general software options include REPLICS 
(REal FLOW Imaging and Classification System), which allows for the real-time 
automated recognition of fish eggs by using a nearest-neighbors algorithm, though 
it has a higher demonstrated error rate (Iwamoto, Checkley, and Trivedi, 2001). 
VisualSpreadsheet© is linked with the FlowCAM and is capable of extracting 
features and creating size spectra of particles. VisualSpreadsheet can also recog- 
nize plankton according to their similitude and easily links to Zoo/Phytolmage 
and Image) for further analysis (Grosjean and Denis, 2007; Schneider, Rasband, 
and Fliceiri, 2012; Spaulding, 2011). Plankton Visual Analyzer (PVA) is a free 
program which counts, sizes, and classifies particles of digital images (Harris 
et al., 2000). PVA is somewhat time consuming but has recently been developed 
to analyze images from VPR. ZooProcess is a free piece of software designed to 
process images from ZooScan but does not allow automatic recognition of particles, 
though Plankton Identifier, a free piece of software, can be used with ZooProcess 
to identify plankton (Gorsky et al., 2010). 

The automated classification of planktonic community images yields valuable 
data about the plankton, though there are high percentages of “unclassified” images 
in many analyses - up to 80% of all images in some samples (Duffy, 2012). Examples 
from the FlowCAM of the main issues associated with automatic classification can 
be seen in Figure 10.4. There have been complications with identifying and 
avoiding marine snow, detritus, bubbles, and other non-plankton particles in 
captured images (Benfield et al, 2007). The presence of aggregates, groups of 
multiple individuals sometimes associated with detritus or organic matter, can also 
complicate analysis. Some individuals may be visible in the aggregate and can be 
identified by manual counting but not semi-automated counting (Figure 10.4b). 
Aggregates reduce the overall number of plankton identified and can skew the 
relative proportions of each taxon present, as some taxa aggregate more frequently 
than others. Most tools for semi-automated planktonic community analysis capture 
2D images, though the plankton move in three dimensions in moving fluid. As 
such, plankton can be imaged from any angle, making it difficult for the machine 
to recognize the geometry associated with each taxon (Culverhouse et al, 2006). 
Movement during capture in the three-dimensional fluid also causes some images 
to contain blurs, which further impede the classifier’s ability to correctly identify 
plankton (Figure 10.4c) (Culverhouse et al, 2006). Additional problems have 
been encountered due to the use of length and width to separate out different 
taxa, as larger and smaller forms of the same species can then be incorrectly 
classified as different taxa (Bell and Hopcroft, 2008). Though there are many issues 
with classification, community analysis has been largely successful using current 
semi-automated classification tools. 
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10.4 

Future Directions 

While there have been many advances in semi-automated planktonic community 
imaging, there are still many directions for future research. The standardization 
of protocols and references, though not complete, should be prioritized. Standard- 
ization of capture methods is not possible, due to the wide variety of sampling 
methods and purposes that currently exist in semi-automated planktonic imaging. 
Standardization of analysis is also not totally possible, but future work should aim 
to unite current options and ensure cross-compatibility in analysis formats. This 
will require the development of open source software that is easily and readily 
accessible. The software would need to be easily adaptable and extremely user 
friendly, a major issue with current software. Future collaboration should focus on 
this area to make sure analyses are useful in future research. 

The high degree of variability in machine learning methods leads to a lack 
of repeatability in many analyses. The bias introduced by taxonomists, the type 
and quality of images used for machine learning, and the various algorithms 
used to develop classifiers cause incompatibility among groups of studies and 
complicates analysis. Comprehensive, global training sets for machine learning 
should be compiled and made readily available for comparison. These would 
reduce the variability of analysis and, if kept online in an easily accessible database, 
could be updated with the discovery of new taxa and species. Future work should 
also focus on developing databases of images collected by planktonic community 
imagers, as these images could allow for interesting and innovative secondary 
analysis. 

The use of morphology should be the main focus for future classification 
systems. By analyzing morphology, as opposed to simply geometry, more accurate 
classification could be made. Classifications could also reveal much more about the 
current status of populations within the community and further enable comparison 
through time (Naselli-Flores, Padisak, and Albay, 2007). Morphology analysis could 
be conducted using many of the same techniques as current facial recognition 
software. As morphology based analysis becomes common, research should focus 
on the development of open source, universal 3D standard models. These would 
allow recognition of the morphology of any taxon from any angle, eliminating some 
of the issues associated with capture in fluids. 



10.5 

Conclusion 

Semi-automated planktonic community classification has enabled many important 
studies on global communities. Semi-automated classification systems reduce the 
time and cost associated with whole community studies, enabling a greater number 
of researchers to study more systems in greater detail than previously thought. 
Each study requires a sampling, capture, importation, and analysis method, each 
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of which should be considered before beginning study. There are several options 
for analysis in each step from sampling to final classification and choices should 
be tailored to the goals and limitations of the study proposed. Close collaboration 
between planktonic community scientists and computer scientists should be 
encouraged throughout the process to ensure compatibility. As computer and 
photographic technology advances and international collaboration is encouraged, 
semi-automatic community analysis may finally become fully automated with a 
focus on morphology, not just geometry and taxonomy. 
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